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In ths light of tho dsniand for grsatsr accountability » cost-bonoflt 
analysis is one of the mcst vital concerns of educators, as well as 
others such as those in management in business and industry. There 
is much talk about cost-benefit analysis, but not many practicloners 
have the time to sift through the mounting literature on the subject to 
extract what is useful in their situations. 

Dr. Sara Steele, on the faculty of the University of Wisconsin, has 
prepared a review of the literature about cost-benefit analysis which 
neither overwhelms the novice nor oversimplifies the procedure for the 
experienced. 

The Intent of this publication is philosophical rather than a proce- 
dural presentation; it is not so much trying to tell the reader how to do 
a cost- benefit analysis, but rather to help him understand what he may 
be doing if he elects to carry out such an evaluative procedure. It pre- 
sents a^ brief overview of the nature of cost- benefit analysis and intro- 
duces the reader to some references which will take him furtiier in his 
understanding. 

The author - Dr. Steele - is an adult educator rather dian a cost- 
benefit specialist. Her philosophical discussion in the mid-section of 
the paper is drawn from 17 years with Extension and Intensive study of 
conceptual aspects of evaluation during the past four years as part of 
teaching a graduate level course in program evaluation at die University 
of Wisconsin. She views cost-benefit analysis as an activity which is 
bodi intriguing and threatening, but about which adult educators need to 
know considerably more if they are to make a good Judgment of whether 
or not it is a tool that will help them in their programming. 

On behalf of die many adult educators who will use this literature 
guide, we thank Sara Steele for the talent and many hours she has de- 
voted to its preparation. We hope diat this review, and otliers of a 
similar namre, will bring the literature of adult education a step closer 
to help improve research and practice In the field. 

We are also gratefultothe Adult Education Association of the U.S. A, 
for making dils review more easily available. 

Stanley M. Grabowskl 
Acting Director, 

ERIC Clearinghouse on Adult Education 



I. COST-BENEFIT ANALYSIS AND THE ADULT 



As an adult educator, are you interested in 
the efficiency of your program? There appears 
to be a widespread trend in other fields of educa- 
tion and related social welfare fields to consider 
not only the effectiveness of programs but also 
the efficiency of those programs. Some would say 
that an efficiency epidemic is in process. The 
germ, which was incubated in industry, swept 
through the government branches most influenced 
by businessmen, infiltrated HEW and, through fed- 
eral funding of local education activities, has now 
become a common phenomenon in many fields of 
public school education. 

There is relatively little evidence of how far 
the efficiency epidemic may be spreading into 
adult education. It has been extensively used in 
some areas like manpower training programs, but 
there is little evidence that It Is having Impact 
upon other areas such as liberal adult education. 
The Handbook of Adult Educatio n,^ pnbliahed in 
1970, has a one- sentence reference to coat effec- 
tiveness buried In a chapter on training in busi- 
ness and industry. Most adult education litera- 



ture on program evaluation still stays with the 
1952 definition that evaluation Is dc ormining the 
extent to which objectives have been attained,*- 
That definition does not Include examining the 
cost of attaining those objectives , 

Some adult educators are probably rabid fans 
of cost-benefit analysis and other forms of effi- 
ciency examination. Manj^ are rabidly opposed. 
Most are in the middle - trying to figure out what 
is involved and how much value cost-benefit 
analysis m y have for them. 

In order to determine value, one first needs 
to get a feel of the concepts that are involved 
and how they are being operationalized In adult 
education. The first section of this paper will 
present a brief overview of concepts of cost- 
benefit analysis; the second v/111 review its cur- 
rent usage; the third will present a somewhat 
philosophical discussion of thu potential of cost- 
benefit analysis for adult edMcatora, We will go 
beyond values and explore a very broad ap- 
proach to •the idea of input-outpUi. anarysis. 



II. COST-BENEFIT ANALYSIS CONCEPTS 



Program Efficiency 

Let’s start with the concept of program effi- 
ciency, which is part of current pushes for the 
input of scientific (l.e., hard, data- supported) 
blinking into everyday organizational decision 
making. Some now define evaluation as the proc- 
ess of providing "information through formal 
means, such as criteria, measurement and statis- 
tics to serve as rational bases for making judg- 
ments in decision situations. "3 

What is programming efficiency? For our 
purposes program efficiency relates to how well 
the program utilizes the resources that are in- 
vested in it. The Glossary of Administrative 
Terms in Public Healldi defines efficiency as "die 
capacity of an Individual, organization, facility, 
operation, or activity to produce results in propor- 
tion to the effort expanded. ”4 Suchman Indicates 
diat ‘ ‘efficiency is concerned with the evaluation 
of alternative padis or mediods in terms of cost — 
in money, time, personnel and public conven- 
ience. ”5 There appear to be two Important di- 
mensions to a concept of efficiency: one is the 
idea of proportion or ratio , tiie otiier is the idea 
of alternative usage . 
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Proportion deals with the relationship of in- 
put to output within a given program without re- 
gard to alternatives. For example, it raises such 
questions as: Were the accom.pllshments suffi- 

cient for the amount of resources expended? Were 
resources fully utilized in the course of the pro- 
gram or was chere waste and slippage? Was the 
value of the result great enough to merit the cost 
of attaining that result? 

The dimension of alternative usage, however, 
compares one proportion of input to output with 
otiher proportions which might be attained. It 
deals wltii such questions as: Is there any better 
way to attain the same results? Can the same 
end result be achieved in different ways at less 
cost? Of these two proposals, which will produce 
most at the least cost to the agency? Would tiiere 
have been a greater value to die participants, to 
the agency, or to society, if the same resources 
had been Inyested in a different program? 

Current Approciches for 
Examining iffitslency 

Along widi die push for analyzing die efficien- 
cy of educatlorial operations has come the concept 
of system and of system analysis. A variety of 
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management activities have a system base and 
focus on efficiency of operation. Three of the most 
common are TERT, PPBS, and cost-benefit 
analysis. Although this paper is focusing pri- 
marily on cost-benefit analysis which deals with 
how efficiently resources are used in producing 
a product, it is important also to see the relation- 
ship of cost-benefit to the other two procedures. 

Cost-benefit analysis is usually a sub-part 
of PPBS, (Program Planning Budgeting System), 
which deals ' itli the overall budget operation. 
This system compares the efficiency of one pro- 
gram proposal with another by means of cost- 
benefit analysis, PPBS differs from other forms 
of budgeting in that it exaniines both resource 
input and the potential results from those inputs. 
Former budgeting approaches focused almost 
entirely on resource inputs. PPBS requires die 
agency to establish clear statements of expected 
outcomes in quantifiable or measurable terms 
and examine alternative costs and benefits in 
carrying out the mission of the agency.^ 

Cost-Benefit analysis and PERT (Program 
Evaluation and Review Technique) often are teamed 
in examinj.j,ig the efficiency of an on-going pro- 
gram, While cost-benefit focuses on the relation- 
ship of investment to results, PERT examines the 
efficiency of the process by which the results are 
attained, 

Ryden calls the procedure one of establishing 
a network of the events and activities Involved 
in developing the program. The network is a 
graphic representation of the project plan, Tlnie 
estimates and deadlines are established for each 
item in the network. Actual operations can dien 
be examined against the proposed network to see 
whether progress is proceeding as expected, and 
whether resources are being adequately used in 
those processes.^ 

System As An Important 
Underlying Concept 

We make better use of tools when we under - 
stand dielr relationship to the phenomena to which 
they are being applied and the circumstances 
which brought a given tool into being. It Is tempt- 
ing to borrow cost-benefit analysis as a tool and 
adapt it to our needs in adult education, consider- 
ing only cost-benefit per se. Our understanding 
will be greater, however, if we take time to con- 
sider that cost-benefit analysis Is a system tool. ° 
It is based on the central premise that the activity 
to which it is being applied is a system , or an 
assemblage of parts united by some form or regr 
lar interaction or interdependence. Componentb 
include both the inputs in terms of men, money, 
o 
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and materials to do the tasks of the system, and 
the nature of the task itself. Those phenomena 
which make up the completed task are tlieouqiuts 
of the system. In addition to Inputs and outputs, 
however, one must also be concerned with the 
boundaries of the system, the component parts 
that make up the system as a whole, and how those 
parts are organized. 

And one must be concerned with the relation- 
ships of various systems -with the idea of sub- 
systems and of super- systems. Most systems are 
responding to the needs of some larger system, 
and the outputs need to be things wanted and 
needed by a super- system. An educational insti- 
tution, for example, is a system, but the nature 
of its ouq^uts are affected by what some other 
system - businesses or industries, the commu- 
nity, or a segment of society - wants. Within the 
educational institution there may be a variety of 
sub-systems at work, such as teams of teachers, 
and back-up personnel like audio-visual special- 
ists. 

In understanding the systemic nature of his 
operation, the educator does well to look at two 
concepts of system, The concept that has been 
oudined above looks at system primarily through 
the eyes of hie engineer or other professional 
dealing with system in term.s of physical or opera- 
tional phenomena,. The second relevant concept of 
system is that of the social system . Each part of 
a system, sub-system, or super-system which 
has people Involved and where these people share 
something in common and interact in some fairly 
regular way, comes to have a social identity with 
Its own people- interaction parts. The nature of 
the social system has been described in various 
ways by sociologists like Loomis, 9 and the I'ela" 
tionship of such systems to education by For est.^^ 
Social systems become important In examining 
program efficiency In diat the various social sys- 
tems involved directly affect the efficiency of the 
physical or operational system that is being used. 

This adds up to complexity and to gigantic 
problems. The systems approach was developed 
as a means of dealing with complexity and com- 
plicated problems. As production tasks became 
more complex and as problems became more and 
more complicated, it has become important to 
analyze die complex phenomena and identify Its 
component parts and die relationships between 
die parts as a means of defining a problem and 
identifying ways of dealing with that problem. The 
systems approach also Is Important In keeping 
complex operations going efficiently by preventing 
problems. 

Cost-benefit analysis Is sometimes extracted 
and applied witiiout regard to the position of inputs 
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and outputs and process to the system, Hosvever, 
it is well to keep the concept of system and its 
importance in mind as we move on to a more 
detailed description of cost-benefit analysis. 



What Is Cost- Benefit Analysis? 

Cost- benefit analysis is a procedure by which 
the cost of producing a product is compared with 
the value of the product that is produced. In its 
most common form, both the cost and the benefit 
are described i\i monetary terms. 

The following descriptions of cost-benefit 
analysis identify the basic ingredients and related 
ideas. 

By “cost- benefit" is meant die relation- 
ship of the resources required (cost) to attain 
certain goals (benefit). It Is based on the 
economic concept that many executive deci- 
sions involve the aliocation (best use) of 
limited resources among competing require- 
ments. The allocation of available resources 
is determined by a comparative analysis of 
the current system with presumably prac- 
ticable alternative systems, ^ 

^ ^ ^ 

“...any analytic study designed to assist 
a decision-maker in identifying a preferred 
choice among possible alternatives , . , usually 
it consists of an attempt to minimize dollar 
cost subiact to some mission requirement 
(which may not be measurable In dollar 
terms) or, conversely, to maximize some 
physical measure of ou^ut subject to a bud- 
get constraint. "12 

sft Iff 

‘ ' Cost- Benefit, or Cost-Effectiven ess, 

Analysis is the quantitative examination of 
alternative prospective systems as to the 
potential trade-cflfs with regard to the bene- 
fits, or effectiveness, to be gained and die 
costs to be incurred among the alternatives 
for the purpose of identifying the preferred 
system and Its associated equipment, product, 
etc. Dollars are die units most generally 
used, but any economic resource might, for a 
given problem, be the limiting factor - e.g„ 

personnel." 13 

The three descriptions vary In terms of the 
complexity of the idea diey present. However, It 
is apparent that there is an emphasis on (1) die 
relationship of the technique of cost-benefit anal- 
ysis to decision making, (2) die technique as being 
an analytical process, and (3) going beyond the 
simple ratio of resources to results and making 
a comparison of alternative ratios, 
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Quade suggests that there are five elements 
of analysis, each of which is present in every 
analysis of choice, aldiough diey may not always 
be explicitly identified: (1) the objective (or 

objectives), (2) die alternatives, (3) the costs, 
(4) a model (or models), (5) a criterion. The 
criterion is die rule or standard by which die 
alternatives may be ranked in order of desirability 
and the most promising may be chosen. It pro- 
vides a means for weighting costs against effec- 
tiveness.!^ 

A cost-benefit analysis usually consists of 
five steps: 

1. Identifying die pertinent measures of 
effectiveness, i.e,, benefits. 

2. Describing the alternatives. 

3. Expressing both mission performance and 
cost as functions of the characteristics 
of each akernatlve. 

4. Estimating appropriate values for die 
equation parameters. 

5. Computing, analyzing, and presenting re- 
sults, 13 

Uses of Cost- Benefit Analysrs 

One part of understanding a concej it is under- 
standing how it can be used, 

Hatry suggests the following uses of cost- 
benefit analysis: 

(1) To aid decision makers, in situations 
where ftinds or other economic resources 
are a significant constraint, in making 
improved choices among alternative sys- 
tems, 

(3) To provide Indications as to specific 
areas for further study or research in 
which ftmds might be applied to bring 
overall system savings or to reduce 
critical areas of uncertainty, 

(4) To suggest, in certain situations, addi- 
tional alternatives tiiat would fare well 
in die face of die major difficulties of the 
examined alternatives and diat, there- 
fore, might be preferable, 

(5) To Indicate, for industrial users, the 
ma^ltude of die potential business for 
major components of die preferred sys- 
tem - at least, for specified effectiveness 

levels, 16 
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Carpenter and Haggarf of the RAND Corpora- 
tion, a company which has been closely identified 
with the concept of cost-effectiveness analysis, 
have indicated the following uses for cost-effec- 
tiveness analysis in educational settings: 

--to help assess the relative worth of several 
innovative programs with the same educa- 
tional outcome. 

--to determine whether a single program is 
becoming more or less effective as time 
passes so that steps may be taken to improve 
it, if necessary. 

--to help assess the relative worth of the same 
program for different student populations. 17 

Like any tool, one must be clear in terms of 
what the cost-benefit analysis can and can’t do. 

Carpenter and Haggart point out that: 

“The goal of the analysis is not to provide 
the planner with the alternative that ’maxi- 
mizes’ or ‘minimizes’ specific characteris- 
tics; the goal is to provide information which 
together with the judgment of the planner 
permits a compromise among the character- 
istics of the alternate few within the various 
environmental constraints, such as budget 
lebel or political atmosphere.’’! - 

Levine cautions: 

“It is not a substitute for the educated 
judgment of the decision maker. Radier, it 
provides a package of relevant information on 
which to basecertainklndsof decisions. Also, 
it does not favor the ‘cheapest’ or even the 
‘best’ program but the optimum program in 
terms of the available resources (money, 
trained personnel, facilities). “19 

Definitions of Costs and Benefits 

As various components are identified and 
clarified in field operations, the Jargon increases. 
For example, experiences in evaluating programs 
in the war on poverty indicate that costs may be 
of two types; direct costs and opportunity costs. 

Opportunity costs consist of the otiber benefits 
foregone; for example, what, if anydiingj would an 
on- the -job- training participant be earning if he 
had been employed elsewhere and not participat- 
ing in this program? 20 

Benefits also have been defined as being of 
two types: private and social. Private benefits 



are those accruing to the Individuals Involved, 
Social returns are returns which accrue to the 
entire society and not to any one individual. 

One important component of a concept of bene- 
fits is that of the attainment of objectives. Each 
person who is maiiagement-by-objectives (or 
education-by- objectives) oriented will have to 
reconcile the degree to wh.ch die attainment of 
objectives is synonymous with the benefits of die 
program. In many instances benefits are defined 
as those changes w'hich are defined in die pro- 
gram’s objectives. Levine suggests that the ob- 
jectives, goals, values, etc., determine the agreed 
upon kind and degree of change. The objectives 
constitute the criteria which determine whether a 
“benefit” is in fact so.^l However, if one adapts 
Seri van’s framework of effects of education to 
beneiits of education,22 it might be suggested that, 
in addition to the benefits directly defined in the 
objectives of the program, additional beneficial 
effects to others than those directly involved in 
the program might also well be considered in an 
analysis of benefits. 

In some instances negative effects may also 
need to be calculated and subtracted from bene- 
fits before the cost-benefit ratio is figitred. 

Problems In Cost-Benefit Analysis 

Cost- benefits analysis of education and social 
action programs Is relatively new. There is much 
yet to be thought through and developed in ordei 
for tile tool to reach its fullest value in these 
settings. 

Even some of the strongest proponents of 
cost-benefit analysis agree tiiat It is not easily 
or automatically carried out. Like many other 
evaluative activities, much of the process Is de- 
pendent upon human judgment. 

Carpenter and Haggart point to some of the 
following problems in using cost-benefit analysis 
in educational situations; difficulty In determin- 
ing the resources required in a program, tiie mis- 
leading nature of a single measure, the lack of a 
fully developed metiiodology for cost analysis, 
dittlculty in measuring effectiveness, tiie inaade- 
quacy of a single measure, the lack of metiiodology 
for estimating the effectiveness of future Pro- 
grams, the need for criteria for effectiveness. 23 

Cain and Hollister give an extensive discus- 
sion of problems in the design of evaluating social 
action programs using a cost-benefit approach, 24 

They discuss problems in specifying objectives, 
using control groups, tiie replicability criterion, 
statistical considerations, economic consldera- 



tions, including the discount rate and organiza- 
tional problems, such as timing and ability to 
hold design and internal data systems. 

It is apparent that, at its present stage of 
development, cost-benefit analysis is like any 
other technlciues of analysis. Operationalization 
of it is based upon certain decisions andassun.p- 
tions. As in other situations, these decisions and 
assumptions often are open to question. The in- 
dividual doing the analysis needs to be able to 
support his decisions and assumptinns. The extent 
of his ability to do so will, in iJart, affect the 
trust that others will place in his analysis. 

Techniquei Associated With 
Cost- Benefit Analysis 

Some of the problems Inherent In operational- 
izing cost effectiveness have led to the use or 
development of some special techniques. Tech- 
niques particularly helpful in cost analysis or 
estimation, which appears to be developing into a 
specialized field of its own, include: (1)^ Linear 
Programming Techniques , such as statistical re- 
gression analysis for estimating costs oi proposed 
items which are generally similar but somewhat 
different from those on which direct cost experi- 
ence is available. A distinctive feature of the 
resulting estimating equatlcns is that they relate 
selected physical or performance characteristics 
of the item to cost, (2) Progress C urve Analysjj 
— for use in estimating future hardware costs in 
those Instances where varying the quantitative 
requirements appears to produce a significantly 
different unit cost, (3) Engineering Analysis , 

to provide the basic information for describing die 
cost- generating characteristics of new or future 
components. 2 5 

One of die major problems in cost-benefit 
analysis is the inherent uncertainty of the data. 
In order to make data more useful, various tech- 
niques have been evolved for testing the degree 
of uncertainty or for identifying range under given 
conditions. In se nsitivity analysis the costs for 
die most extrerne variables are plotted and die 
niost critical variables identified. Usually, costs 
are plotted according to die high cost and low 
cost conditions. 

A second companion technique Is that of 
subiective conf idence limits. In diis approach 
die range of values for each parameter is assunied 
to conform to a particular probability distribution 
and thus to Indicate likelihood of the various 

situations occurring.26 , 

Variations of Cost- Benefit Analysis 

As a tool is used in new settings, new adapta- 
tions of it emerge. Tanner uses die term utility- 



cost sensith ty analysis in a somewhat different 
way. He makes cost-benefit analysis more useful 
in situations where it is hard to affix dollar values 
by suggesting thatvalue judgments based on some- 
thing (need, value, output, etc.) have usually al- 
ready been made about school activities and tiiat 
what needs to be done is to translate them into a 
common numerical scale of utility factors which 
will show comparative values of die various ac- 
tivities. 

In this procedure all of the activities to be 
compared are assigned a utility factor which 
shov/s their rank In terms of order of importance 
and preferences. These utility factors become the 
numerators for their respective activity costs and 
the cost-utility ratio is then calculated. These 
ratios provide an index to aid decision makers in 
allocation of resources. The crucial step in this 
process is assigning some type of numerical 
value of benefit which is not so much a value of 
the benefits to the activity itself as it Is a symbol 
of how those benefits rank in comparison with 
odier activities which are being examined. 

Carpenter and Haggart suggest that there may 
be value in educators constructing equal- cost 
alternatives as a part of cost-benefit analysis. 
In such an approach, the costs of all programs 
would be held at the same point through adjusting 
the dimensions of the program, such as the number 
of students enrolled. With cost held constant, the 
comparison becomes focused on differences In 

effectiveness. As an alternative, equal- effective- 
ness alternatives might be constructed where the 
output is held constant and suggested programs 
are examined In terms of variation in input. The 
latter requires that it be possible to calculate 

output accurately,28 

Another approach has been developed by a 
public health team. They suggest that ffiere are 
tiiree basic components of programs: Objectives 
(O), Activities (A), and Resources (R). To deter- 
mine program efficiency, they suggest that the 
attainment of die program objective which Is at- 
tributable to die activity (AO) should be compared 
with die desired level the planners had^expected 
to result from die program (PO), ihe ratio 
AO:PO describes die effectiveness of the program. 
They define efllciency as the ratio between the net 
attainment of objectives (AO), ouqput, and ^e 
program resources expended (AR), input (AO: AR), 
The reverse ratio AH: AO yields a measure of 
average cost. They suggest that efficiency studies 
may answer several relationship questions: 

1) relationship of objectives attained to re- 
sources ei^ended, AO:AR or AR;AO; 



2) activities performed to resources ex- 
pended, AA.'AR or AR:AA; 

3) objectives attained to activities per- 
formed, AO: A A or AA:AO, 

To some extent the EMIS system (Extension 
Management Information System), which has been 



adopted by the entire Cooperative Extension Serv- 
ice, is attempting to examine some of these rela- 
tionships. Timels the major resource examined 
both in terms of activities performed and objec- 
tives attained. The system has an added dimen- 
sion of examining the time one has planned as 
compared with the time that was actually spent 
on the various objectives. 



III. COST-BENEFIT ANALYSIS OF ADULT EDUCATION 



Some cost- benefit analysis of adult education 
programs is being attempted. However, If the 
reports in the ERIC collection are any Indication, 
it would appear that the use of cost-benefit tech- 
niques has been limited primarily to 1) programs 
which lend themselves to achieving economic out- 
puts, and 2) programs which are closely associated 

with industry. 

Over half of the 70 items indexed as cost- 
benefit analyses in the ERIC collection on Janu- 
ary 1, 1971, dealt with studies of training pro- 
grams. Many were reports of On-The-Job Train- 
ing programs or otiiers carried out under the 
Manpower Development Act, Some examined 
otiier types of industrial training programs. About 
ten percent dealt with die cost-benefits of farm 
management education programs, anodier ten 



percent dealt with studies of cost-effectiveness 
of audio-visual programs or otlier programs using 
educational technology. Three studies dealt with 
cost-benefit analysis of general adult education 
programs (Utah, Quebec, and Massachusetts), one 
with adult basic education, one with Title 1, and 
one Nvith teacher decision-making. About a tenth 
of the references were general references to tlie 
concept of cost-benefit and its use in educational 
planning. 

The adult educatorproposing to do cost-bene- 
fit analysis will do well to study the techniques 
used in some of these studies and to consult with 
those who did the analysis and who gained a greater 
understanding of the procedure, potentials, and 
difficulties through first-hand experiences. 



IV. INPUT-OUTPUT ANALYSIS AND ADULT EDUCATION 



Cost-benefit analysis is beginning to be used 
in adult education. It is a system's tool which 
has had considerable value in other settings. What 
is its potential value to die adult educator? 

Although a value analysis may be of use 
to some programs at some points in time, a 
greater value may lie in broadening die concept 
to one of input-ou^ut analysis and considering 
some of the philosophical questions diat may be 
involved in such analysis. 



This final section of die paper will consider 
die following questions* (1) Are we dealing with 
a system? (2) When is input- output analysis im- 
portant to the adult educator? (3) At what depth, 
stage, and level should such analysis be done? 
(4) What are the key decisions In operationalizing 
input- ou^ut analysis in adult education? 

Are We Oealing With a System? 



Fundamental to the concept of input-ou^ut 
analysis is the basic concept of system. Is adult 
o 'ucatlon (the educational program you are exam- 



ining) a system? Should you view it as a system 
when you attempt to understand what you are doing? 

A few years ago tiiere may have been an 
occasional argument as to whether adult education 
was an art or a science. Now it seems that the 
question might be — Is it an art, a science, or a 
system? 

It is possible that adult education is all three. 
Our difficulties may arise from not recognizing 
and utilizing die qualities of all three that are 
present. Adult education, or a given program- 
ming operation, is a system In that tliere are 
clearly identifiable parts united by some form, 
regular Interaction, or Interdependence, We know 
enough about the process and some of the com- 
ponents to be able to deal wldi tiiem as with a 
science. And yet widi die range of complexity 
Involved wldi human variance, diere is still a 
great deal of art Involved in adult education. 

In odier words, it may be extremely helpful 
for us to view our operations as we would a 
production system and to clearly analyze the parts 



and their relationships. But at the same time we 
must be constantly aware that in die coirmlex 
human interactions which are involved in educa- 
tion, regardless of our care in planning and in 
acting, we cannot be sure that Z will automatically 
follow if we apply X to Y in a certain way. The 
surety which is usually operating in manufactur- 
ing systems or other activities dealing primarily 
with non-human elements just isn’t there In adult 
education, unless v/e resort to techniques re- 
sembling brainwashing. 

Frustrating as the complexity of the situation 
is, one might argue that the potential for variance 
is a good thing - that adults need to be free to Inter- 
act with the educational stimuli in their own way, 
based upon their Individual needs and individual 
experiences. Certain inputs may result in certain 
results in terms of certain students, but to de- 
mand that those inputs result in the same results 
for all students may not often be appropriate. 

Yet it can still be valuable to analyze our 
work as though it were a production system if we 
don't act as though it is a mechanical, semi- 
automatic process. 



When I'i input -Ouput Analysis 
Important to the Adult Educator? 

In addition to looking at adult education as a 
system, it may also be helpful to do Input-output 
analysis. Fach of us has to decide why we want 
to do such an analysis and when it is most appro- 
priate. 



Some educators may have to do input-output 
analysis (in this case cost-benefit analysis); in 
some instances they may need to do such studies; 
in odier Instances they may want to make sure 
of a particular technique as part of their process 
of decision making or as a means of gaining a 
better understanding of the process of adult educa- 
tion. 

The have to pressures may come from at least 
two sources - programs directly supported by the 
federal government, and situations where military 
or industrial experts (or odier power figures com- 
mitted to the concept of cost-benefit analysis) are 
on local boards or in administrative positions 
and are able to exert strong pressures for tills 
kind of analysis. In these instances, tiie adult 
educator may have to be prepared to carry out, 
or have a fairly formalized process of cost- 
benefit analysis applied to his program. 



Adult education may as a whole be more 



o 



exempt than other branches of education from tiie 
impulsion to apply such techniques, because 



many parts of adult education are so structured 
that at least partial control rests directly in the 
hands of local citizens and there is direct visible 
accountability to those who are providing the 
money to support the program. Pressures for 
cost- benefit analysis arc very apt to come from 
more remote funding sources. The self- support- 
ing program which is not relying on tax monies 
or otiier monies, and remote either geographically 
or in interest from the program, may be less apt 
to feel pressure for using this particular tech- 
nique, because judgments in terms of input-output 
ratio (in tiie sense of whether the value received 
exceeds the cost) are being made daily by those 
enroillng in and continuing with adult education 
programs. 

Need to -- Input-output analysis does not be- 
come important to many adult educators until a 
given resource is scarce and there is a good deal 
of need for it, or a good deal of competition to 
secure it. At that point, the adult educator may 
be willing to expend extra effort in order to utilize 
the amount of the resource that he is able to mus- 
ter in the most efficient way possible. 

There may be some adult educators - those 
who have unlimited resources, or few pressures 
on those resources, and who have similarly blest 
clientele — who can luxuriate in being inefficient. 
However, if any shortage is Involved, then effi- 
ciency rears its head (ugly or enticing depending 
upon how it is handled). 

Want to - Some adult educators may want to 
make Input-output analysis an important tool of 
programming. 

If an adult educator is strongly influenced by 
tiie Protestant ethic, a good frugal up-bringliig, a 
strong business motive or any of the related value 
syndromes, eHlciency may be Important for its 
own sake. If so, tiie adult educator will be con- 
cerned with proportion and best use of resources 
even though tiie resources are suflicienfly abun- 
dant to meet present and projected future needs. 
He will be efficiency-orientated because efficiency 
Is an Important element in his value system, 

Anotiier type of adult educator who may at 
least want to test out input-outi*ut analysis Is tiie 
Individual who has an “Itch to know” more about 
tiie adult education process. Examining ou^ut 
under varying input situations can tell us a good 
deal more about how to optimize learning for 
different types of clientele in different situations. 

For tiiese two types of adult educators and 
for the individual who uses input-output an^ysis 
as a tool in handling a scarce resource, Input- 



output analysis can become a viable tool rather 
than a threat. 

At What Depth, Stage and Level 
Should Analysis Be Done? 

The kind of input- output analysis which Is 
most often found In the literature involves a fairly 
complex process, utilizing sophisticated I'esearch 
techniques, which is applied at the end of some 
major program. As a result we are apt to have 
a fairly rigid concept of input- output evaluation. 
As a tool, however, input-ouQDut analysis can be 
extremely flexible if we are creative enough to 
make it so. It can vary in depth, be applied at 
different programming stages, and be applied at 
each level of programming. 

Depth - Various continuums have been built 
exploring the degree of depth of evaluation. Usu- 
ally the continuum has at one end the type of 
evaluation that takes place in everyday thought 
processes and at the other some description of a 
highly formalized process which draws heavily 
on research procedures. There may be as much 
value in applying input- output analysis at the 
everyday end of the continuum as there is in 
applying it in the formalized study sense. As we 
choose between alternatives in everyday activities 
in adult education, it may be helpful to consider 
both what we may need to invest in each alterna- 
tive and what we might expect as results from it. 
Although we may be more accurate in our con- 
clusions if we can marshal accurate data to help 
us make comparisons, we may sometimes have 
to use the data which can readily marshal and 
discount it on the basis of apparent limitations 
rather than careSjlly designing a study-type ap- 
proach with substantial fresh data collection. 
Sometimes we just can’t wait for the formalized 
study. We have to draw upon past experience and 
what little we know about the new situation. One 
way of Improving use of data at the point in need 
may be that of developing Information systems 
which will have the relevant data needed for an 
input-ou^ut analysis readily at hand at die point 
in time when we need to make a decision. Too 
often, evaluation is diought of as a point- in-Ume 
evaluative study designed to secure data rather 
dian an ongoing process utilizing data which is 
produced by a carefully designed continuous data 
system. The system analysis approach tends 
toward die type of design which continually pro- 
vides high quality data for die types of decisions 
diat need to be made when diat decision arises. 

Stage in Programming — Evaluative actilvides, 
including cost-benefit analysis, have traditionally 
been associated with the conclusion of a program. 
?^«.vever, PPBS calls for cost-benefit analysis to 
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be applied to a program before the program is 
approved for the coming year. Decisions between 
Drograms are based upon prediction as well as 
past performance. Input-ouT»ut analysis can be 
as important a tool In designing a program - 
choosing between alternative approaches to be 
used “ as it is in summative evaluation. The con- 
cept of proportion between resources and expected 
results and alternative usage are important at 
each stage in die programming process. 

Level of Programming — Cost-benefit anal- 
ysis has tended to be introduced as a tool of top 
administration. However, the concept of Input- 
ou^ut analysis and its related activities may be 
as important at the very micro level (individual 
teacher-participant Interactions) as it is at the 
very macro level (the top executive making na- 
tional budget decisions which may affect literally 
hundreds of individual programs.) An action in 
the most micro sphere has an input and an output 
and a proportion of return. In most Instances, 
there are alternative ways that the ou^ut could 
have been secured or alternative uses to which 
die input could have been put. This is a part of 
the operational decision making that is the life 
blood of programming and guiding learning. The 
dimensions of both proportion and alteinatlves 
also pervade all odier levels of programming, 
aldiough at higher levels more attention may be 
paid to die alternative proportions than to the 
proportion of any one program. 

What Are the Key Decisions? 

The basic decisions which will give thr pro- 
grammer the most difficulty as he attempts to 
operationalize a concept of input-ouirut analysis 
in terms of his program are not diose dealing 
widi how to quantify or how to measure and 
analyze; there are ways of surmounting problems 
in diese areas. The basic decisions which must 
be grappled wldi and where the programmer will 
find few easy and clear answers Include* (1) which 
resources? (2) whose resources? (3) ou^ut for 
whom? (4) what ou^uts? (5) what is value? 

In each Instance, he wlU have to deal wldi 
the philosophical as well as the practical Issues 
involved. If he Is willing to buck what litde tradi- 
tion diere Is and be creative, he has considerable 
choice open to him In regard to each of diese 
points of decision and can choose what Is most 
relevant to the needs of the situation. However, 
In each case he will be vulnerable to opposition 
and must have his rationale firmly in mind. As 
he makes his choices and continues on his way 
exploring die relationship of input to output, he 
not only must be Independent and innovative, but 
he also must continue to check the reliability and 
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validity ’of his decisions so that he doesn’t end up 
expending considerable resources on a house-of- 
cards that can be leveled by a few concise sen- 
tences from a clearheaded opponent. But the 
decisions rest with him. As with any other evalu- 
ative tool, input-output analysis is a tool for im- 
proving and facilitating human Judgment, but can- 
not ever replace it. Its quality rests primarily 
on the human judgment that controls it. That 
judgment is a part of the professional qualifica- 
tions of the really top-notch adult educator. 

Which Resources - Aldiough we automatically 
think of the money Involved, other types of input 
may also need to be scrutinized. 

In periods of a tight economy, both the par- 
ticipant and the adult education agency may be 
concerned about money. The potential program 
participants may have heavy demands competing 
for the money which they would like to use for 
more education. The agency can’t produce nearly 
as many programs as It would like, so it has to 
concentrate on those expected to have maximum 
value. Even “society” as a whole becomes in- 
volved when there is heavy pressure on the tax 
dollar or other dollars earmarked for social 
betterment and those dollars could be spent In 
many ways in improving the welfare of the popu- 
lation. 

Regardless of whether money is scarce, an- 
otiier resource that must be taken into considera- 
tion is time-bodi the time of the programmer 
and the time of the participant. Both can invest 
tiieir time in a variety of ways. Which way will 
give diem better payoff for the amount of time 
invested? 

Still another resource which Is not usually 
found in lists of resources, but which is a key 
one in die programming situation, is diat of 
human energy . This resource includes not only 
physicaT energy but also mental energy used In 
creativity and deep diought. Successful educa- 
tional activities require energy Input from bodi 
the programmer and from the participaints. To 
borrow the McClusky power/load concept, ^0 this 
energy at times becomes a limited resource and 
may need to be used efficiently. There are odier 
resources, of course — personnel, supplies and 
materials, facilities — which may be scarce and, 
therefore, require efficient utilization and which 
at times need to be the focus of input- ouff)Ut 
analysis. 

Whose Resource Inputs? The last section 
suggested die Importance of analyzing an adult 
education program from die standpoint of die par- 
ticipant as well as from the standpoint of the 



agency, There may be at least four owners of 
resources that need to be considered in input- 
output analysis - the adult educator, his agency, 
the participant, and the broader society. 

The adult educator is concerned about those 
of his own resources which he invests - his time, 
his energy, his creativity, the intensity of his 
attention, etc. Unless he is functioning on his 
own, he also must be avmre of the use of the 
agency resources - total manhours, materials, 
facilities, supplies, etc. The agency and the in- 
dividual resources are apt to rank uppermost In 
terms of many adult educators’ contemplation of 
efficiency. 

But adult education has an important element 
that is different from most other “production” 
situations. Its consumers are volunteers. There- 
fore, the way diey view the use of their own re- 
sources will become an Important factor In terms 
of how much they will participate. Does a given 
program appear to require too much from them 
(money, time, energy, etc.) for what they expect 
to get out of it? Or, more importantly, do they 
see a strong enough value resulting from their 
participation? Does a given activity widiln a pro- 
gram require so much that the Individual will 
mentally or physically drop out? It is possible 
for a program to utilize the programmer’s re- 
sources and those of the agency relatively effi- 
ciently but to be extremely wasteful in die use 
of the participants’ time or energy. Most adult 
educators have come to recognize that many of 
their clients do have certain limits to die re- 
sources diey can Invest and to take these limits 
into account as they develop programs. For 
example, “A $10 fee may be too high for die ex- 
pected clientele;” “Students in a night class have 
been on their feet for 12 hours and may have low 
energy levels; ” or “F arm homem^ers have to be 
back home by chore time,” However, adult edu- 
cators may have given lass attention, as they at- 
tempt to figure efficiency ratios on programs, to 
calculating die ratio from the participant’s point 
of view - how much did he have to put into the 
program (time, energy, etc.) as compared with 
how much he feels that he got from the program. 

The fourth owner of resources, sometimes 
forgotten by die adult educator, is “society.” In 
one respect the time, money, and energy of all 
die individuals Involved is part of a “societal 
pool.” One use of die resources in that pool may 
mean diat diose resources are not available for 
oMer uses,, For example, when die modem middle- 
aged matron is engrossed in formal continuing 
education courses, she Is not spendhig diat same 
time on social causes and volunteer work. So- 
ciety may be losing something when she ei^ends 



her energy elsewhere. Granted, it may be gain- 
ing something else through her better undeptand- 
ing of contemporary situations. But which has 
tile most value at a given time and in the long run? 

Output for Whom? - Just as four separate but 
related entitles make Input Investments into each 
adult education program, so the same four - the 
adult educator, the agency, the individual, and 
society -may all be concerned with the output. 

Basically we might say the most important 
output is that which the participant receives and 
the value of such output to society which reflects 
back on die image of the programmer and die 
agency. But at times, all four may have separate 
desires In terms of outjiut. For example, the out- 
put wanted by die participants and die output 
needed by society may not always be the same. 
In die case of programs dealing with etiinic rela- 
tions, the white conservative may want some- 
diing diat will help him ease Ws conplence, while 
society wants decisive positive action from him# 
Participants may take part in other programs to 
increase their own self-concept and confidence 
with no intent of using what they gain from a pro- 
gram, Society may want greater production from 

diem. 

Realistically, survival and image enhance- 
ment are important objectives of the agency and 
of the programmer although we seldom are honest 
enough to put those particular cards face up on 
die table. Sometimes the agency and the program- 
mer will invest Inputs for the primary purpose of 
maintenance radier than die desired outputs of 
some segments of society, or some segment of 
die clientele. Or, in somewhat different terms, 
the agency is usually Interested In outputs which 
enhance Its reputation in die eyes of tiiose who have 
power over the particular agency. 

In addition, die creative, energetic program- 
mer is concerned with output which gives him 
satisfaction. According to his own values and 
interests, he may find some results of programs 
much more satisfying than some others. 

Just as there are several sources of input 
which, in relation to a given decision, maybe im- 
portant for analysis by die adult educator, so 
also diere may be benefits to several sources. 
In some analyses, it is Important to look at die 
output to a given source in comparison widi die 
input invested by that source. For example, what 
does the participant gain for what he Invests? 
What does die programmer get from what he in- 
vests of himself and his skUl? What does i^e 
agency secure in terms of image and security 
from its invesanent in a program? What does 
o 
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society get in terms of ihe betterment of its con- 
ditions by its two-fold investment - through die 
agency costs of the program and through the costs 
in participant time and money? 

In other instances. It may be important to 
look at the benefits to one of the investors which 
come from the invesdnent of one of the others. 
For example, how does society benefit from the 
investments of die participants? How does the 
participant benefit from the Investments made 
by the agency? Although a given input-output 
analysis may concern itself with only one set of 
resources and one set of resurts, occasionally a 
programmer may want to think through the mesh 
of interrelationships between the inputs from 
the four sources and die outputs to those sources. 

What Outputs? — What are the ou^uts that 
each of die four expect? How can they be cate- 
gorized? 

Some adult educators would say that die 
answer to die question, “What output should be 
expected from a program?" is simple. “Of 
course, it’s the extent to which the program 
meets Its objectives," Others would say that in 
adult education the ou^uts are changes in knowl- 
edge, skill, and attitude. Some would view this 
kind of output in almost die same way that they 
would view the output of formal school youth edu- 
cation, except, of course, that tiie clientele are 
adults and, as such, certain differences in ap- 
proach must be utilized. Some would be very 
apecific in labeling the precise behaviors ex- 
pected from a given program. 

But is the identification of oupput quite that 
simple? Should one be concerned only with the 
extent to which certain learning objectives are 
attained? Some Hterature on evaluation points 
out that programs may produce botii direct effects 
and side effects and diat the effects may be short- 
and long-range. Programs may produce both 
positive and negative results, often yielding a 
blend of die two, 

Houle’s explorations of reasons for learning 
would tell us that not all who participate are seek- 
ing specific knowledge, skill, or attitudes.^^ If 
dlls is die case, dien we might hypodieslze that 
die socially-oriented participant may desire quite 
different output from his participation than does 
die task-oriented participant. 

In some instances, there may be much more 
concern wldi broad development, eidier on die 
part of die individual, or tiirough him on the part 
of a community or segment of society, which can 
be stimulated, directed, influenced, or speeded up 
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through participation in adult education pi'ograms. 
This may be a more important output than the 
output related to the specific learning objectives 
of a given program. In agricultural development 
programs, for example, Hayes suggests diat 
changing farmers’ attitudes about using new prac- 
tices and continuing to learn new methods maybe 
a more valuable result than farmers simply adopt- 
ing one new practice in a given program. 32 

There may be a wide range of outputs tliat 
one or more of the four types of participants 
might desire. Some include* 

1. economic Impru’/ement - outputs which 
show marked gain in the economic status 
of the participants, the community, or the 
economy of a region or country, 

2. action - ou^uts which result in direct ac- 
tion on die part of die participants. 

3. specific knowledge, attitude or skill - in 
some instances knowing may be enough, 

4. development - “better” people, groups, 
or communities because of participating 
in adult education programs. “Better” 
may be defined in several ways - more 
open to ideas, more productive, more 
skillful in everyday relationships, better 
communicators, more affluent, more at- 
tractive, more comfortable. 

5. enhanced self-concept — a feeling of 
“worth” and ability to proceed with con- 
fidence. 

6. satisfaction and enlotunent - the pleasure 
of learning and/or interacting widi other s, 

7. prestige - the acmal being among diose 
pYesent may yield a value above and be- 
yond what actually happens while one is 
present. 

The educator is faced widi some philosophical 
dllemm.as as he recognizes some of die different 
kinds of ou^uts that the various investors may be 
expecting. He soon realizes diat the results of 
his programs may not be quite so easy to define 
as he once diought they were. Of course, he may 
choose, to confine himself to education only in the 
sense of helphig his participants add to tiielr 
knowledge, gain skill, or clarify tiheir attitudes 
and develop some affective orientations, but is 
that all die program will be judged by? Is it all 
it should be judged by? 

The alert educator considers what outcomes 
each of die parties involved may see as most 



important - the participants the agency, society, 
and the educator himself. Then he has to decide 
what he is going to do with conflicting expecta- 
tions. His response moves from the realm of 
philosophy to a recognition of any political con- 
notations that may be involved. 

Both the philosophical and the political con- 
notations must be c ‘^'Tdered too, when one is 
doing an Input-ou^u analysis of a program. If 
society is mterested in die analysis, it (or the 
leaders that represent it) may not be satisfied 
with the analysis unless the benefits are defined 
in the way they desire. The militant black, for 
example, may want benefits defined in terms of 
Increased self-concept on the part of the black 
and reduced racism on the part of the white, and 
may not be satisfied wldi an analysis done on the 
basis of benefits in terms of acquiring facts. If 
the power figures representing society have the 
power to support or to wash out the program, 
dien it becomes important that the analysis be 
made on their terms. 

The Value Questio n — As with other forms of 
evaluation, input-output evaluation may ultimately 
lead die adult educator to face the concept of 
value. What is valuable to whom? What is most 
valuable? How valuable are the results in com- 
parison with die value of the investinent? WHAT 
IS VALUE? HOW DO YOU DEFINE OR DESCRIBE 
IT? 

Our typical approaches to evaluation and 
training in research give us little preparation for 
dealing wldi basic value questions. And yet diey, 
in die long run, are die heart of evaluation. The 
basic questions, not only in efficiency evaluation 
but in all decision making in programming, ulti- 
mately deal wldi comparative value of die alter- 
natives, Over die years we have ap’^roached such 
problems by saying that we need more scientif- 
ically pure data. Sclentlflcallypure data, however, 
has little utility unless it Is applied in situations 
where someone is able to assign or relate it to 
value components. The adult educator cannot 
escape valuing by burrowing into data; as a pro- 
fessional he must be able to define and assign 
value and use data, statistics, and die tools of 
sophisticated research to help him in this task, 
but bare numbers seldom provide die most im- 
portant answers. 

If any firm answers are to be found as to die 
nature of value, they are as apt to come from the 
philosophers as from the economists. Or, prob- 
ably, die adult educator needs to draw from a 
wide range of disciplines as well as from his 
own common sense and experience as he tries to 
define die value components involved in his pro- 
gram. 



In Conclusion -- It ■ ould appear that, beyond 
coping With cost- benefit analysis in a monetary 
sense when it is required, the concept of input- 
output analysis may have other values in im- 
proving adult education programs. Perhaps these 
values He not so much in “answers’* which the 
tool can help to provide as in the philosophical 
questions related to resources, their use, and 
value of output that the jprogrammer may have 
to deal with as he develops the framework for 
his analysis. 

Here, cost-benefit is like other evaluative 
activities in that the real value may lie not half 
so much in the “hard data" that such an analysis 
may produce as in the nature of the thinking 
that is triggered and the better understanding of 
that complex phenomenon - modern adult educa- 
tion. 

Little time has been devoted in this paper to 
the processes involved in carrying out input-output 
analysis procedures. We know much more about 
the processes of producing data than we do about 
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Evaluative Research (Russell Sage Foundation, 
New York, 1967), pp. 64-5. 

5. Ibid. 

6. Joseph H. McGlvney, “The New Systems’ 
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A Second Look," Educational Technology, IX, 
No. 12 (December, 1969), pp. 31-34. 

For additional references on PPBS, see: 
Hartley, Harry J., Educational Planning-Pro- 
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the processes of using data. There is a danger 
that the average fledgling researcher (and most 
adult education Ph.D.’s are fledgling researchers) 
vflll become so enamored with data production 
that he falls to deal with the philosophical issues 
that are involved. Data and tlie processes which 
produce it are tools rather than ends. 

Hard data (scientifically processed informa- 
tion) is seldom hard enough to stand unchallenged 
and be accepted as conclusive proof in contro- 
versial situations. However, for years we have 
deluded ourselves into a will-o’-the-wisp search 
for such proof. With our engrossment in sophis- 
tic'ated research techniques, we have minimized 
the other half of the equation. Data has no value 
unless it is used. The challenge of input-output 
analysis and other forms of evaluation is not 
merely obtaining scientifically pure (reliable, ob- 
jective, valid) data, but also using that data in a 
way that has value. It is only then that the benefits 
of evaluation will equal the costs Involved, There 
lies the untapped powerhouse. 
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COST-EFFECTIVENESS ANALYSIS: NEW AP- 

PROACHES IN DECISION-MAKING. Goldman, 
Thomas A., Ed. Washington Operations Research 
Council. 249 p, 1967. Available from Frederick 
A. Praeger, Publishers, 111 Fourdi Avenue, New 
York, N.Y. 10003. 

Since refinements in cost effectiveness anal- 
ysis in the past few years have made it highly 
acceptable as an aid in decision making at high 
policy levels, a detailed discussion of some 
aspects and implications of cost effectiveness 
analysis are included in this book. The first five 
chapters give an overview; measures of effective- 
ness, choice of analytic techniques, use of cost 
estimates, and estimating systems costs. Chap- 
ters Six, Seven, and Eight give examples of mili- 
tary uses for cost effectiveness analysis, while 
chapters Nine, Ten, and Eleven show applications 
for government domestic programs, the War on 
Poverty, and metropolitan transportation sys- 
tems, The last two chapters cover structures of 
incentive contracts and of military industrial 
funds. Extensive tables and references are In- 
cluded, 

THE METHODOLOGY OF EVALUATING SOCIAL 
ACTION PROGRAMS, Cain, Glen G.; Hollister, 
Robinson G. Wisconsin Univ, 22 p. Nov, 1969, 
ED 044 611, EDRS PRICE MF 650, HC $3,29. 
Also available from Industrial Relations Research 
Institute, 4315 Social Science Building, 1180 Ob- 
servatory Drive, Madison, Wisconsin 53706, Re- 
g-Dinted from Public-Private Manpower Policies. 
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Focusing on die evaluation of outcomes (‘ ‘cost 
benefit analysis”) In large scale social action 
programs, this paper examines issues relating to 
the adequacy of theory and mediodology as well 
as the Impact of different types of persons (aca- 
demics, politicians, program adm.inlstrators) in- 
volved in the evaluation process. Problems of 
evaluation design - control groups, the criterion 
of replicability (applicability on a wider scale), 
statistical analysis, and implications of differing 
socioeconomic criteria -are considered in detail. 
The audiors then propose a deliberately experi- 
mental approach which would permit program 
planners and administrators to learn faster by 
testing alternative concepts simultaneously. Using 
an analogy with the court system, they also discuss 
die potential value of a “rules in evidence” ap- 
proach to setting standards for acceptance of 
evaluation results. The document includes 19 
footnotes, folio v/ed by reprints and odier publica- 
tions by die Industrial Relations Research Insti- 
tute and the Center for Studies in Vocational and 
Ifechnlcal Education, 

EDUCATIONAL PLANNING; THE WORLD YEAR 
BOOK OF EDUCATION, 1967, Bereday, George 
Z, F., ed,; Lauwerys, Joseph A,, ed, 554 p, 1967, 
Harcourt, Brace, and World, Inc., 737 Third Ave,, 
New York, N.Y. 10017, 

The 1967 world yearbook on education con- 
tains 25 articles on die dieory of educational 
planning, the major current approaches (social 
demand, manpower requirements, and cost effec- 
tiveness) to educational planning, manpower and 
educational planning in underdeveloped areas 
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(specifically, Africa and Latin America), and the 
creation and use of theoretical models. Relation- 
ships between educational and economic planning, 
historical trends and present patterns in educa- 
tional expenditures, educational and manpower 
planning in several of the major Industrialized 
nations, the Mediterranean Regional Project of 
the Organization for Economic Co-operation and 
Development (OECD), urban unemployment in 
Nigeria, primary education and rural employment, 
the training of planners, and rates of return on 
educational investments, are among the topics 
treated. The document includes an index, chapter 
references, and 79 tables, charts, and figures. 

A THEORETICAL STUDY OF THE DEMAND 
FOR EDUCATION. Onlki, Hajime. Stanford 
Univ., Calif. 115 p. 1968. University Micro- 
films, (Order No. 69-8234, MF $3,00, Xerography 
$7.20) "^h.D. Thesis, 

In this study of the economic process of edu- 
cation, Individual activities regarding education 
are treated as a process of overtime resource 
allocation. The optimum plan for receiving edu- 
cation is investigated in terms of a mathematical 
model, and the demand for education is derived 
by examining the effects of changes in parameters 
on this optimum plan. First, a theory is formu- 
lated which concentrates on how to spend one's 
time between receiving education and engaging in 
work, and how to divide one's income between 
paying for education and saving. Costs are ana- 
lyzed as Income spent and wages foregone; bene- 
fits are defined as improved labor efficiency and 
personal satisfaction. The model is then analyzed 
to reveal characteristics of a person’s plan for 
receiving education. It is shown, among other 
tilings, that one generally receives education be- 
fore engaging in work, ^at demand for education 
decreases when funds are scarce or when educa- 
tional costs rise, and that rising wages can in- 
crease or decrease educational demand accord- 
ing to whetiier the wage shift is proportional or 
constant at each educational level. 

INFORMATION SOURCES AND COMMUNICA- 
TION CHANNELS RELATED TO FARM PRAC- 
TICES ADOPTION IN CENTRAL INDIA. Sharma, 
D. K,; Leagans, J, Paul, N.Y, State Univ,, Ithaca, 
Coll, of Agriculture at Cornell, 31 p. 1968. 
ED 041 210, EDRS PRICE MF 650, HC $3.29. 
Available from Department of Education, New 
York State College of Agriculture, Itiiaca, New 
York /'S^-’gle Copy free; additional copies 250 
each). 

T . dy’s main purpose was to identify and 
describu the roles played by various Information 
sources and communication channels used by 
Sehore farmers in the adoption of four Improved 
■ cultural practices in U.P, District of India, 
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A sample of randomly picked 200 farmers was 
interviewed by questionnaire. Of the five infor- 
mation sources studied, neighbors were named 
by all respondents, village level workers by 72%, 
village chairmen by 26%, and agricultural and 
university personnel by 20% and 17% respectively. 
Among the .seven means of communication, film 
demonstrations led by 50%, followed by radio 
with 38%, audio-visual aids, general meetings, and 
reading materials. Five adoption stages were 
recognized: awareness. Interest, evaluation, trial, 
and decision to adopt. Age, education, caste and 
economic status were discriminating factors, 
singly or in combination, in tiie use of all sources 
except neighbors and village local workers. One 
major finding was that face to face contacts be- 
tween agents and farmers had the greatest influ- 
ence, and mass media the least Influence on the 
adoption, 

THE PLANNING AND COST-BENEFIT ANALYSIS 
OF SOCIAL INVESTMENTS. (In INTERNATION- 
AL INSTITUT E FOR LABOUR STUDIES BUL- 
LETIN. 2/4-47, Feb. 1967). Scott, Norman. 
Feb. 1967. 

This article explains cost benefit analysis, 
newest of the techniques for assessing the effi- 
ciency of public expenditures. Whereas social 
benefits are excluded from ordinary financial 
analysis of Investment, cost benefit analysis 
traces effects over tiie entire period of reper- 
cussions on the economy and therefore on present 
and future social welfare, to reconcile private 
good witii social good and private cost witii social 
cost, so Impormnt where the state is increasingly 
assuming economic responsibilities. General 
techniques and main characteristics of economic 
planning are discussed, and a macro- economic 
planning model devi,sed as a 15-year plan for 
Poland is described. Questions answered are 
what to include as costs and benefits, how to value 
them by establishing shadow prices and opportunity 
costs, which discount rates to use In reducing 
future values to present values for comparing 
present costs, and what technical and aoclAL 
constraints tiie planner meets. Case studies 
related to management. Industrial relations, and 
manpower training present cost benefit analyses 
of manpower retraining in West Virginia, retrain- 
ing in Connecticut, and rehousing 500 Indian fami- 
lies in South Dakota. Emphasized are implica- 
tions for economic planning and development of 
generalized use of cost benefit techniques to 
apply to sector and national planning levels, 

COST-EFFECTIVENESS ANALYSIS FOR THE 
“WAR ON POVERTY.” Besen, Stanley M.; and 
others. 15 p. 1967. Chapter 10 in Cost-Effec- 
tiveness Analysis, edited by Thomas A, Goldman. 
Available from Frederick A. Praeger, Publish- 
ers, New York, N.Y, 10003. 
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Two anti-povorty programs — investment in 
education and in highways and other public facili- 
ties — should increase earnings and employment 
through an increase in production possibilities and 
a fuller utilization of existing resources. In 
evaluating training programs, there is consider" 
able evidence that training expenditures have a 
return at least equal to that on other forma of 
capital investment; the return has both private and 
social components, and there are direct and op- 
portunity costs to consider. In a Massachusetts 
retraining program from 1958-61 the “net bene- 
fits accruing to society” were evaluated as about 
$3300 per worker; a similar study In West Vir- 
ginia showed net returns of about $4000 per 
worker. The Job Corps and the Neighborhood 
Youtii Corps cannot be judged properly by study- 
ing returns to other training programs, since 
these two programs differ significantly from 
others. New highways have benefits in real out- 
put and employment, new industry, and an increase 
in land values. There are many questions relat- 
ing to cost analysis In highway construction which 
cannot be answered adequately at present. Until 
programs can be evaluated effectively, large scale 
financial commitments should not be made; pro- 
grams should be adequately diversified and steps 
for evaluation should be set up. 

Adult Education— Broad Range Programi 

REVIEW AND SYNTHESIS OF RESEARCH ON 
THE PLACEMENT AND FOLLOW-UP OF VOCA- 
TIONAL EDUCATION STUDENTS. Little, J. 
Kenneth. Ohio State Univ„ Center for Vocational 
and Tfechnical Education. 54 p, Feb, 1970, ED 
037 543, EDRS PRICE MF 650, HC $3.29. Avail- 
able also from The Center for ’^''ocatlonai and 
Technical Education, Ohio State University, 1900 
Kenny Rd., Columbus, Ohio 43210, ($1.75), 

This re’sdew of follow-up studies of graduates 
of vocational- technical education programs in- 
cludes studies published since 1965 and covers 
the major fields of vocational -technical education 
at secondary, post secondary, and adult levels. 
Programs for special groups of Individuals are 
also included. Two purposes of the review are 

(1) to identify the job histories of graduates and 

(2) to provide baseline data for program evalua- 
tion. The review is organized around titie follow- 
ing topics: The Role of Followup Studies, General 
Surveys, Studies of Specific Programs, Summar- 
ies of Placement Information, Studies of Special 
Groups, Benefit-Cost Studies, Followup Studies 
Procedures, and Evaluation Summary, A 101- 
item bibliography Is included, 

RENTABILITE DBS INVESTISSEMENTS EN 
EDUCATION DES ADULTES; ASPECT SOCIAL 
T^HE RETURN ON INVESTMENTS IN ADULT 



EDUCATION: THE SOCIAL ASPECT). Jobin, 

Ghyslaln. Laval Univ. (Quebec) 10 p. 1968. 
Text in French. 

This study investigated the return on public 
investments in adult education for the Province of 
Quebec as well as greater Montreal and Quebec 
City. Emphasis was on the cost effectiveness of 
part time and full time formal study at ages 19, 
24, 34, 44, and 54. It was found that returns on 
Investments favor full-time over part-time study; 
tiiat cost effectiveness increases with age and with 
educational level; and that promoting higher adult 
education in the Montreal area would prove espe- 
cially advantageous. 

A TEACHER RESOURCE ALLOCATION DECI- 
SION-MAKING PROCEDURE FOR AN EVENING 
INSTRUCTIONAL PROGRAM OF A CALIFORNIA 
PUBLIC JUNIOR COLLEGE. Tayloe, Ralph 
Chester. California Univ,, Los Angeles, 129 p, 
1967. University Microfilms, (Order No, 67-11, 
708, MF $3.00, Xerography $6.20). Ed.D. Thesis. 

A study was made of the specific cyclical 
sequence and detailed events needed to allocate 
available teacher resources to evening division 
courses at a California public junior college. 
Using data from interviews with participants and 
from odier sources, a flow chart model was made 
of the allocation procedure. Sequential activities 
were described as a part of one of three major 
segments of the total process: determining 

foundation courses; assigning resources to such 
courses; and controlling course enrollment. Find- 
ings Included the following: (1) foundation courses 
receive tije teacher resources, and are generally 
the same as courses offered in die last similar 
semester; (2) foundation courses grow to serve 
community needs by adding specialized courses or 
new class sections; (3) average relative instructor 
cost per student (Spring 1966) was 20,66 mills 
after six low enrollment classes were cancelled; 
(4) at the end of the six- week enrollment period, 
86% of total capacity was in use, effective cost 
averaged 24 mills, and departmental costs ranged 
from 17.75 to 45.87 mills per student hour for each 
dollar of Instructor cost, 

CONTiNUING EDUCATION: STATE PROGRAMS 

FOR THE 1970'S. Levin, Melvin R.; Slavet, 
Joseph S. 157 p. 1970, Available from Raydieon 
Education Company, D.C. Headi and Company, 
475 South Dean St., Englewood, N. J, 07631, Re- 
port prepared for the Massachusetts Advisory 
Council on Education. 

Varying needs, largely unmet, were found in 
die provision of adult education diroughout Massa- 
chusetts, and recommendations were formulated 
for an articulated and adequate program. Low 
priority, weak centers of leadership, and a 
limited and confused mandate emerged as major 



obstacles to effectiveness. It was found that, on 
the basis of accepted cost benefit criteria, con- 
tinuing education programs tend to cost relatively 
little and pay off rather well. Moreover, the 
potential market for basic education, high school 
equivalency, citizenship education, occupational 
training, driver education, staff development and 
career development for government persoiinel, 
and other forms of continuing education far ex- 
ceeded current levels of activity. A policy decla- 
"ation was suggested, followed by guidelines for 
program planning and student recruitment, ad- 
ministrative and organizational changes (includ- 
ing creation of a new division), financial and 
other measures to strengtlien the statewide con- 
tinuing education system, a network of public 
relations officers, inservice and other training 
for state employees, and scholarships for con- 
tinuing higher education. The document includes 
an index, a bibliography, 227 chapter references, 
and seven tables, 

A SUMMARY OF ACTIVITIES IN TENNESSEE 
UNDER TITLE I OF THE HIGHER EDUCATION 
ACT OF 1965. SECOND PROGRESS AND EVAL- 
UATION REPORT. 129 p. Nov. 1968. Tennessee 
Univ., Knoxville, State Agency for Title I, ED 
027 461, EDRS PRICE MF 650, HC $6.58. 

An evaluative and historical report is pre- 
sented of 38 university extensioii programs pro- 
posed in Tennessee during Fiscal Years 1966, 
1967, and 1968, under Title I (Higher Education 
Act of 1965), together with a list of 13 programs 
completed prior to June 30, 1967. The Univer- 
sity of Tennessee, East Tennessee State Univer- 
sity, Memphis State University, Tennessee Tech- 
nological University, and 14 other colleges and 
universities are represented by a total of 87 pro- 
gram involvements in such areas of community 
action and service as professional continuing 
education, leadership training and updating of 
municipal officials, identification of community 
problems, public health education, urban planning 
and development, law enforcement, regional eco- 
nomic development, youth opportunities, guidance 
and counseling, and training in adult education. 
Recommendations and explanatory comments on 
scheduling of Federal appropriations, new sources 
of revenue, development of institutional capacity, 
problem identification, program development and 
activity, communication with adult lay leaders, 
evaluation procedures, and indirect cost deter- 
mination are offered by the State Agency for Title 
I as a plan for future action, 

UTAH ADULT EDUCATION SERVICES. ADULT 
EDUCATION REPORT 1968-69. 39 p. May 1970. 
Utah State Board of Education, Salt Lake City. 
ED 042 941, EDRS PRICE MF 650, HC $3.29, 
Major purposes for the preparation of tiiis 
^’'eport on public school adult education in Utah 



were: to provide the public widi a description of 

achievements, trends, and needs, and with mean- 
ingful cost accounting information; to make com- 
p 3 j"isons and analyses of adult education by pro- 
gram, school district, and year; and to provide 

die adult education personnel with needed program 
management and cost accounting information. The 
topics of discussion include: general adult educa- 
tion; adult basic education; estimation of program 
resources available for fiscal year 1969-1970; 
eflective use of program resources; adult high 
school completion; civil defense education; and 
general educational development. Statistical 
tables are included. 

EFFECTS OF TEAM ARRANGEMENT ON TEAM 
PERFORMANCE, A LEARNING-THEORETIC 
AN ‘ LYSTS fin Journal of P ersonality and Social 
Psychology , 3(5)/541-550, 1966). Egerman.Karl, 
10 p. 1966. 

Three groups of six two-man teams, differ- 
ing only in arrangement, underv/ent two major 
phases of training — preteam, where each individ- 
ual developed a proficiency in making a timing 
response, and team training, where each subject 
used his timing skill as a team member. Individ- 
ual preteam proficiencies and the team arrange- 
ment were the only two variables used to predict 
(1) initial team performance, (2) the schedule of 
reinforcement for each subject, and (3) the manner 
in which team performance would change from ^e 
initial to the final periods of training. This in- 
vestigation points out the feasibility of applying 
learning theoretic principles to a study of group 
behavior, The document includes two tables and 
21 references. 

Audio Visual 

THE IMPORTANCE OF TRAINING REQUIRE- 
MENTS INFORMATION IN THE DESIGN AND USE 
OF AVIATION TRAINING DEVICES. Prophet, 
Wallace W. George Washington Unlv., Washing- 
ton, D.C. Dec. 1966, 

Research is reported on determining die 
value of present synthetic flight training, and 
developing programs of instruction to make opti- 
mum use of available devices used In army heli- 
copter training. Three equivalent groups of pilots 
received different training; the trainer group in 
an expensive sophisticated cockpit trainer, die 
mock-up group in a less sophisticated plywood 
cockpit, and the third, in die aircraft. All groups 
were tested on die aircraft, wltii the mock— up 
group doing as well as die trainer group in all 
procedures. It was concluded that beneflwidj. 
training can be accomplished in cruder devices 
that may lack in physical fidelity for successful 
transfer of training. For efficiency and economy 
of training and increased flight safety, it is sug- 



gested that training devices should be constructed 
after analysis of training requirements, which are 
based upon psychological rather than engineering 
needs; and synthetic training programs should be 
tested for effectiveness, and provide an effective 
means of reinforcing infrequently occurring be- 
haviors. 

MORE ON PUBLICITY. Selby, David. In Adult 
Education (London); v42 n5 p. 304-10, Jan. 1970. 

THE NEW MEDIA -MEMO TO EDUCATIONAL 
PLANNERS. Schramm, Wilbur; And Otliers. 
United National Educational Scientific and Cult. 
Org. A-2315. 1967. Available from UNESCO, 

Place de Fontenoy, 75 PariS“7E, France ($2.95), 

This study reviews the educational uses, ef- 
fectiveness and costs of the new media (radio, 
television, programmediearning, correspondence 
study, and films and audio-visual aids) andmakes 
suggestions towards dieir effective use in educa- 
tion, new media have been used for upgrading 
iistruction, teacher training, extension of educa- 
tion, literacy and fundamental education, adult 
education and community development. Generally, 
they have been effective, depending partly on the 
use to which they have been placed, program con- 
tent, technical and organizational support, and 
die context in which reception takes place. Initial, 
operating, and expansion costs are variable. While 
the new media entail higher total costs, economies 
of scale make low unit costs possible, as a guide 
to planners, check list of 17 key elements related 
to planning, organization and support, production, 
reception, and feedback Is presented and dis- 
cussed. Emphasis is laid on die distinctions be- 
tween production and reception and among tech- 
nical components, content and teaching, support- 
ing communications and evaluation, and training 
provisions. Planners are urged to take a systems 
analysis approach and to view the new media as 
efficient system components which may be used to 
meet a recognized educational need, 

SECOND SEMINAR ON DIRECT TEACHING BY 
TELEVISION rSCHEVENINGEN, 1968). 48 p. 

1969. Council' of Europe, Strasbourg (France). 
Council for Cultural Co-operation. ED 039 590, 
EDRS PRICE MF 650, HC $3.29. 

A seminar on direct teaching by television, 
held In Rome In December of 1966, created such 
interest that It was decided to call a second semi- 
nar, restricted to adult education, which was held 
at Schevenlngen, July 1-5, 1968. The subject was 
die assessment of needs and evaluation of results. 
The purpose was to compare, at a European level, 
the metiiods already developed in diese two fields, 
pool experience, and promote coordination be- 
tween educational authorities and radio and tele- 
vision experts. Many kinds of research can be 



useful to the assessment of needs, but special 
stress should be put on the study ot society, cul- 
ture, and personality in one system of inter- 
dependent relations, functions, and values. Re- 
search into evaluation of radio and television 
programs must have as its aim to make sure 
that the needs met have been correctly assessed, 
that the means employed for meeting needs are 
the best, that targets have been reached, and that 
the operation is warranted in terms of cost- 
effectiveness. There is need for a pilot project, 
at national and international levels, on evaluation 
of the results of a series of experimental educa- 
tional television programs. 

A COST ANALYSIS OF MINIMUM DISTANCE TV 
NETWORKING FOR BROADCASTING MEDICAL 
INFORMATION. Dei Rossi, J. A.; and Others. 
84 p. Feb. 1970, ED 038 988, EDRS PRICE MF 
650, HC $3.29. 

Two specific applications for networks might 
be used to serve biomedical community: (1) net- 
working of the nation’s educational television 
(ETV) stations for occasional or one-time broad- 
casting and (2) networking of the nation’s medical 
schools for continuous broadcasting. These two 
applications are analyzed in detail, A second con- 
trmution of diis analysis is the provision of data 
and methodology for examining cost and effective- 
ness (In terms of physicians and students within 
broadcasting range) of configurations of subsets 
of points in the full network. The data and method- 
ology could also be used for determining the maxi- 
mum cost for networks using media other than 
broadband television broadcasting, such as audio 
only broadcasting in conjunction with slides or 
still pictures. The cost attractiveness of net- 
worked, siiTiultaneous broadcasting can also be 
compared with, for example, sequential broadcast- 
ing using mailed video tapes, 

DIRECTED PRIVATE STUDY fin Industrial Train- 
ing International 2(8)/320-322, Aug. 1967). Jones, 
Lyndon H, 3 p. Aug, 1967, 

Directed Private Study (DPS) has evolved 
from correspondence study, providing some face- 
to-face teaching to offset the loneliness of die 
student. New t 3 fpes of courses have been de- 
veloped combining a period of residence with 
independent study, such as the two-year corre- 
spondence study course in madiematlcs at Moscow 
University, die Institute of Management (Kolodje, 
Czechoslovakia) seven-month boarding and slx- 
mondi correspondence course, and the program of 
die Japan State Broadcasting Corporation. An 
experiment at die University of Wisconsin, **Ar- 
ticulated Instructional Media,” in its first pro- 
gram, in library science, provides seminars on 
campus, independent swdy, and library projects. 
Detailed statistics on the comparative costs of 



full-time oral instruction, correspondence 
courses, and Directed Private Study are not avail- 
able, nor are relative examination successes. 
However, costs shown take no account of oppor- 
tunity costs, which tip the balance strongly for 
DPS schemes if the examination results are com- 
parable. Schemes of DPS should perhaps make 
use of programmed and taped materials. We 
must keep much of this work in the public sector, 

NEW EDUCATIONAL MEDIA IN ACTION— CASE 
STUDIES FOR PLANNERS -I. United Nations 
Educational Scientific and Cult, Org. 206 p. 1967, 
Available from UNESCO, Place de Fontenoy, 75 
Paris-7E, France (A. 2285, $4.50). 

This volume contains six cases on the use of 
television, radio, and correspondence study- 
alone or in combination — for Instruction, Educa- 
tional extension, and adult education and com- 
munity development in American Samoa (ETV), 
USA (Hagerstown ETV), Thailand (Educational 
Radio), Japan (ETV and Correspondence), and 
Australia (Radio and Correspondence). Most 
caccG were prepared by a team Including usually 
an educator, media specialist, and economist, who 
presented die history, organization, operation, 
costs, and evidence of effectiveness for each proj- 
ect. In each case, tables and graphs present die 
evidence available to the effect that learning does 
take place and that the new media, while they en- 
tail higher costs, can bring down unit costs, 
depending on how many people are reached. Dif- 
ficulties in planning, organization and support, 
production, reception, and feedback are noted, 
discussed, and pertinent recommendations pre- 
sented. Not unexpectedly, developing countries 
tend to have more difficulties - these are traced 
to technical and financial constraints. They also 
tend not to benefit from the economies of scale in 
the use of the new media. 

PROGRAMED AV PAYS ITS WAY. Johnson, Lowell 
R. In Training in Business and Industry ; v7 nl 
p. 36-39. Jan. 1970. 

The Kohler Company found diat programed 
instruction can effectively reduce Job training 
time. Comparison statistics demonstrated the 
programs more than repaid their costs. 

Farm Managtment Education 

ATTITUDES TOWARD ENTREPRENEURIAL BE- 
HAVIOR AND EDUCATION— THEIR RELATION- 
SHIP TO INSTRUCTION. FINAL REPORT, Kittle- 
son, Howard M, 133 p. Sept, 1969, Minnesota 
Univ, ED 034 894, EDRS PRICE MF 656, HC 
$6,58, 

To evaluate the non- economic returns to 
investment of resources in education, a study was 

conducted to determine the effect of farm manage- 
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ment instruction on attitudes toward education and 
farming, relationships existing between attitudes 
and farm income, and whether smdents' attitudes 
became more like their instructor’s with increas- 
ing years of instruction. The groups in study 
were: (1) farm families who submitted records 

for analysis in the 1967 Minnesota Farm Manage- 
ment Program, (2) farm families who had dropped 
out of the program later than 1964, (3) farm fami- 
lies selected at random from 10 counties, and 
(4) agriculture instruction responsible for farm 
management programs. The 70 per cent response 
to the 90-item attitude Instrument revealed a gen- 
eral Improvement in attitudes related to more 
years of instruction, with a strong suggestion of 
diminishing marginal returns. Families in other 
than well- organized programs had scores nega- 
tively related to instructors; in well-organized 
programs scores were positively related. All 
differences in attitudes were vary slight. There 
Is a need for a revision of metiiods, subject 
matter, or both if adult vocational agriculture 
Instruccors want to claim significantly- improved 
attitude as a product of tlielr educational program. 

THE DEVELOPMENT OF A MODEL DESIGN TO 
ASSESS INSTRUCTION IN FARM MANAGEMENT 
IN TERMS OF ECONOMIC RETURNS AND THE 
UNDERSTANDING OF ECONOMIC PRINCIPLES. 
Rolloff, John August. 155 p. University Micro- 
films, (67-6364, microfilm $3.00, Xeroxed copy 
$7.80). 

The records of 27 farm operators participat- 
ing in farm business analysis programs in five 
Ohio schools were studied to develop and test a 
model for determining the iniluence of the farm 
business analysis phase of vocational agriculture 
instruction in farm management. Economic re- 
turns were measured as ratios between 1965 pro- 
gram inputs and outputs determined by change in 
net farm Income between 1964 and 1965. Data 
collected Include: (1) the relative change in under- 
standing of profit maximizing economic principles 
IndlcatsQ by die change in test scores, (2) the 
change in farm Income indicated by selected eco- 
nomic efficiency factors, (3) the association be- 
tween understanding of principles and die eco- 
nomic efficiency of farm operators, and (4) die 
ratio between input cost of Instruction and eco- 
nomic returns expressed as net farm income. 
In pilot trial die test group showed: (1) An im- 
proved mean change in test scores, (2) increased 
volume and economic efficiency in the farm busi- 
ness, (3) a positive association between farm 
operators’ understanding of profit maximizing 
economic principles and changes in their eco- 
nomic efficiency, and (4) $53,16 increase in net 
farm income for each dollar expended by farmers 
on instruction. Refinement of the model proce- 
dure was projected as a means of advancing 

24 
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inquiry into the micro-economic assessment of 
investments in agricultural education, 

EVALUATION STUDY OF FARM AND HOME 
MANAGEMENT PROGRAM INNEV/ YORKSXME. 
A six-part report. Alexander^ Frank D, 238 p. 
New York State Univ.j Ithaca. Coll, of Agricul- 
ture at Cornell Univ. ED 018 714j, EDRS PRICE 
MF 650, HC $9.87. 

In 1954 Congress earmarked funds for a 
Farm and Home Management Program to offer 
intensive on-the-farm counseling in management 
and decision making. This six-part evaluation of 
die Program as it was conducted In New York 
State, primarily widi dairy operators, was de- 
signed to determine the program’s effectiveness 
relative to less intensive programs. Part 1 
establishes the representativenesB of 250 partici- 
pants in the evaluation sample by comparing them 
to 150 nonparticipants on such matters as size 
of herd and farm receipts. Part 2 is the agricul- 
tural and home demonstration agents’ description 
of the program as reported on questionnaires. 
Part 3 examines the program through the eyes of 
administrative staffs. A case study reveals the 
amount of exposure of 25 participating families 
to 58 dlff rent topics in the program, such as 
analimis of farm expenditure and agronomy. A 
study of a three-township area opens to question 
die hypotiiesis diat the neighborhood automatically 
ftinctions as an Information diffusion channel, A 
cost-benefit study using the agents’ time input as 
major costs and the farm operators’ changes in 
labor incomes as benefits, indicates diat the aver- 
age labor income advantage of 87 participants in 
1959 exceeded the cost of the program, (Tables 
provide statistical data,) 

THE CHANGE AGENT AND THE PROCESS OF 
CHANGE. RESEARCH BULLETIN 17. Nielson, 
James. 100 p. 1967. Michigan State Univ. ED 
029 180, EDRS PRICE MF 650, HC $3.29. 

As a final chapter in the research to evaluate 
the Impact of the Michigan township extension 
experiment, this report focuses on why and how 
change came about, what factors appear to be most 
closely related to change, and die ways in which 
change agents can most effectively help bring 
about change. In die experiment, extension agents 
worked intensively with farm families in five 
township areas for five years; one of the major 
objectives was to develop and test techniques for 
Increasing agricultural oupput. It was found diat 
farmers in experimental areas adopted Improved 
farm practices faster and stepped up their use of 
inputs more dian farmers in matched control 
groups. Average value of total farm output In- 
creased $5,600 on experimental farms as com- 
pared to $3,400 on control farms. Net farm earn- 
ings for die experimental sample increased an 
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average of $1,646 as compared wldi $938 for the 
control sample. Benefit-cost analysis showed that 
the program cost $117 per farm per year. The 
differential increase in net earnings during the 
five-year experimental period exceeded die total 
costs of the program by two and one-half times. 
Included are 32 tables, 14 references, and a model 
of relationship among types of variables studied. 

RETURNS TO HUMAN AND RESEARCH CAPI- 
TAL, UNITES STATES AGRICULTURE, 1949-1964. 
Flshelson, Gideon, 121 p. North Carolina State 
Univ. University Microfilms, (Order No. 69-7 656, 
MF $3,00, Xerography $6,00) Ph.D, Thesis, 

This study estimated rates of return to public 
investments in human and research capital (formal 
schooling and extension and vocational agricul- 
tural education) in the United States agricultural 
Industry, (Southern states were excluded because 
of demographic and educational factors that would 
have biased the variables.) Output per farm was 
defined in terms of human capital, research at 
state agricultural experiment stations, andaiputs 
of land, labor, machinery, otiier expenditures, 
and fertilizer. The Dg distribution of research 
was identified with the rate of adoption of hybrid 
com in com belt states. Values of marginal 
product ranged from, $0.25 to $0.61 per dollar 
per year ^schooling), $4.96 to $19.86 (extension 
and vocational agricultural education), and $5,81 
to $15.28 (reBaarch), Ranges for Intemal rates 
of return were 9.6% to 61%, 196% to 636%, and 
24.6% to 43.1%, refapectively. Conclusions justi- 
fied additional investments in these kinds of educa- 
tion and research, 

AN ECONOMIC STUDY OF THE INVESTMENT 
EFFECTS OF EDUCATION IN AGRICULTURE, 
FINAL REPORT. Persons, Edgar A andOtiiers, 
175 p. Apr. 1968. Minnesota UnF% ED 021 989, 
EDRS PRICE MF 650, HC $6,58, 

To determine die absolute economic return 
to adult farm bueiness management education, the 
diminishing marginal return effect from added 
Increments of education, and benefit- cost ratio of 
the educational program for participants and the 
sponsoring community, data were collected from 
3,578 farm business records representing farm- 
ers enrolled in farm business management educa- 
tion in 1959-65. Farmers participating in die 
educational program were generally younger, 
better educated, and more affluent than ^e aver- 
age farmer described in census data. Polynomial 
curvilinear regression statistical techniques de- 
scribe the relationship between investment in 
farm business management education and diree 
measures of economic success: farmer’s labor 
earnings, return to capital and family labor, and 
total farm sales. Farmer's labor earnings and 
farm sales Increased rapidly during die first three 
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years of instruction, declinsd for the fourth, fifth, 
and sixth years and then increased rapidly in sub- 
se 'ent years. The benefit- cost ratio for indivld- 
uu. farmer participants was 4,20:1. The commu- 
nity benefit- cost ratio when increased business 
activity was measured by increased farm sales 
was 9,00:1, This study assists In establishing a 
rationale for educational programs to increase 
community assets and describes an educational 
model for efficient program operation and growdi. 

INVESTMENTS IN EDUCATION FOR FARMERS: 
SUMMARY OF AN ECONOMIC STUDY OF THE 
INVESTMENT EFFECTS OF EDUCATION IN 
AGRICULTURE. Persons, Edgar A„ and Others. 
54 p. Jan. 1968. Minnesota Unlv. ED 021 951, 
EDRS PRICE MF 650, HC $3.29. 

To investigate the value of an educational 
Investment for farmers and the community, a 
study was Initiated to examine the relationships 
between costs and benefits. An InstructionaTpro- 
gram in farm business and resource management 
from 1959-65 was considered as the educational 
investment, and benefits were assessed by exam- 
ining 3,518 business records of enrolled farmers, 
A benefit-cost analysis revealed: (1) A farmer 

can expect to realize about $4 of labor earnings 
for each dollar of Investment in educational pro- 
grams in farm business and resource manage- 
ment, (2) Where the benefits to the community 
were calculated as an aggregate rise In farm 
labor earnings and the cost included the aggre- 
gate cost borne by the community the ratio was 
approximately 2:1 where the cost included farm 
sales as a measure of business activity the^ratio 
was 9:1, (3) Diminishing marginal return effects 
were observable In educational investments, (4) 
Over tiie 6-year period the cost effectiveness of 
die educational program was not constant, and 
(5) A rigorous, highly structured, and goal di- 
rected educational program in farm business man- 
agement proved to be an excellent educational 
investment for die communivles In diis study, 

OJT and MDA 

THE EVALUATION OF SELECTED PRESENT 
AND POTENTIAL POVERTY PROGRAMS. Muth, 
Richard. Institute for Defense Analyses, Arllng- 
tlon, Va. 91 p. Jan. 1966. Available from NTIS 
(AD-682-203, MF 650, HC $3.00). 

The study attempts to evaluate the efficiency 
of two types of alternative programs aimed at 
Increasing the incomes of poor families. It deals 
with programs involving education and training as 
well as those which represent direct measures 
for increasing family income. In order to com- 
pare such t^rograms, a common measure of effi- 
ciency is needed. The one chosen here is the 
annual cost to the Federal government of raising 
o 
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income by $100 per year until the worker reaches 
age 65. A rate of 9% is used to discount future 
Income. In evaluating education and training 
programs, th^ criterion applied in Judging bene- 
fits is not the change In employment but the in- 
crease in the workers’ future earnings. In the 
absence of adequate data on the benefits of educa- 
tion and training programs. It was assumed that 
Head wStart or nine months of MDTA or Job Corps 
training would yield the same average increase 
in earnings as an additional year of high school 
education. The document Includes 22 tables, with 
appendixes on Federal cost factors and the deter- 
mination of family income. 

A CRITIQUE OF COST-BENEFIT ANALYSES OF 
TRAINING (In Mondily Labor Revi ew, 90(9)/45- 
51), Sewell, David O, 7 p. Sep. 19"57, 

Tills edited version of a critique of cost- 
benefit analysis illumines the difficulties en- 
countered in measuring the returns from on-tiie- 
job training accruing to the individual, society, 
and government. Its hypothesis, that findings 
from Studies on the Manpower Development and 
Training Act projects do not transfer to the War 
on Poverty, supports a request for funding a cost- 
benefit analysis on training the poor under tffe 
Manpower Improvement Through Community 
Effort project in Norffi Carolina, It criticizes 
generalizations that have been based upon three 
published analyses of training schemes conducted 
under the MDTA Act of 1952, the Area Redevelop- 
ment Act of 1961, and earlier State equivalents. 
It points up such difficulties as (1) treating popu- 
lations of the poor and unemployed as die same, 
(2) considering diat improvement In income or 
employment are not die only benefits, and (3) try- 
ing to calculate the value of, complementary de- 
mands for labor created by using the newly trained, 

AN EVALUATION OF INSTITUTIONAL VOCA- 
TIONAL TRAINING RECEIVED BY AMERICAN 
INDIANS THROUGH THE MUSKOGEE, OKLA- 
HOMA AREA OFFICE OF THE BUREAU OF 
INDIAN AFFAIRS. Blume, Paul R, Oklahoma 
State Unlv. 261 p. May 1968. University Micro- 
films. 

The adult vocational training received by 
Indians under Public Law 959 was evaluated to 
determine benefits of Institutional training pro- 
grams and to develop manpower policy recom- 
mendations, Criteria of the evaluation included 
employment experience, income, labor force at- 
tachment, and benefit cost ratio. Of the approxi- 
mately 670 Indians who received adult vocational 
training, information was obtained for 220 from 
school files and by mailed questionnaire. Some 
trainee characteristics were; (1) The average 
trainee was better educated dian die average 
Indian, (2) The employment and Income levels 
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were low by most standards, (3) There was a 
fairly high noncompletion rate among the trainees, 
and (4) There was some indication that the ques- 
tionnaire response rate was partially affected by 
the cultural and historical background of die 
trainees. Conclusions were: (1) Completion of 

training results in an average increase in income 
of $1,929, (2) The average increase in employ- 
ment was about 3 and one half months of additional 
employment, and (3) The social benefit cost ratio 
was found to be 2*39* 

TRAINiNG THE UNEMPLOYED* Somers, Gerald 
G. (Document not available fromEDRS*) Chapter 
11 in IN AID OF THE UNEMPLOYED, edited by 
Joseph M. Becker, 25 p. 1965* Available from 
The Johns Hopkins Press, Homewood Campus, 
Baltimore, Md, 21218* 

Recent government t-- alning programs (largely 
under the Area Redevelopment Act of 1961 and 
die Manpower Development and Training Act of 
1962) are discussed. By examining results of 
early experience under Federal legislation, some 
of d ie conflicting views on the value of these pro- 
grams are assessed. An account is given of the 
content, methods, and procedures of retraining 
surveys in West Virginia and elsewhere, with 
emphasis on job placement, training- related job 
vacancies, selection, trainee characteristics, 
attitudes of nontrainees and dropouts, retraining 
as an Inducement to mobility, and die relation to 
need and to subsequent employment. Company 
and union programs, collective bargaining, and 
Federally subsidized on die job training, and re- 
training in Canada and Western Europe are re- 
viewed, Last to be discussed are the gains and 
costs of investment in human capital. It is con- 
cluded that existing public and private programs 
must be expanded and integrated with odier labor 
market policies and must entail constant retrain- 
ing and upgrading, relocation allowances, and im- 
proved placement and guidance services, 

THE ECONOMIC EFFECTIVENESS OF RE- 
TRAINING THE UNEMPLOYED, A STUDY OF 
THE EXPERIENCE OF WORKERS IN CONNEC- 
TICUT* Borus, Michael E, Yale Univ, 228 p. 
June 1966, (Ph.D. thesis) (Research report to 
Federal Reserve Bank of Boston, 35). ED 010 
865, EDRS PRICE MF 650, HC $9.87. 

A study was made of retraining programs in 
4 labor markets in Connecticut, under both Area 
Redevelopment Act and state sponsorship. The 
purpose was to determine die benefits and costs 
to workers, government, and the economy. The 
state had pioneered in such programs so a sample 
could be studied of diose who had long work his- 
tories after retraining. The trainees were divided 
into six groups-— those who completed the course 
with and without employment, those who withdrew. 



widi and without employment, and those who re- 
fused retraining, with and without employment. 
Three control groups were trainees who, without 
employment, completed the course, wididrew, or 
refused it. The benefit cost ratio for the individ- 
uals (such as reduced annual unemployment and 
increased annual wage vs. income lost during 
training and higher Income tax afterwards) was 
not as high as the benefit to the government (such 
as reduced costs of unemployment and public 
assistance), and to the economy (the increase in 
gross national product and reduced aggregate un- 
employment level). It was felt that the beneficial 
effects of the retraining programs might be low- 
ered by the addition of the Manpower Develop- 
ment and Training Act provision for training the 
hard core unemployed, 

THE EVALUATION OF GOVERNMENT PRO- 
GRAMS: THE CASE OF NEW HAVEN’S MAN- 

POWER TRAINING ACTIVITIES. Goldfarb, Robert 
S. 260 p, 1968, Yale Univ., Ph.D. Thesis. Avail- 
able from NTIS (PB-182-173, MF 650, HC $3.00). 

This thesis attempts to deal with difficult 
methodological problems of program evaluation; 
the particular programs considered are two types 
of manpower training programs: institutional or 
‘ ‘classroom” training and government- subsidized 
on-the-job training. Characteristics of cost- 
benefit analysis are investigated and the method 
is critized for use in analyzing manpower pro- 
grams, Two data sources are used to study the 
production process. First, a group of trainees 
who underwent each t>TS of training were inter- 
viewed. Second, employers who participated In 
the OJT program were Interviewed, as were a 
group of &e largest ernployers in the New Haven 
area, 

MANPOWER TRAINING: SOME COST DIMEN- 

SIONS, Young, Stanley, 51 p* n.d, Massachu- 
setts Univ*, Amherst* Labor Relations and Re- 
search Center. ED 027 390, EDRS PRICEMF 650, 
HC $3.29. 

Some of tile dimensions of the relative f inan- 
ely contribution of tiie cooperating parties in 
manpower institutional training as established un- 
der the Manpower Development and Training Act 
of 1962 were explored* This analysis will provide 
some perspective to tiiose who must finally decide 
tiie question of relative financial contribution, or 
provide them wltii a certain range of the cost 
dimensions of tiie program. Discussed in this 
document are: (1) some cost dimensions of man- 
power training, (2) total cost of tiie manpower 
development program, (3) space, equipment, pro- 
gram planning mid on- ^e- job Institutional train- 
ing, (4) indirect cost, (5) cost requirement, (6) 
individual educational return, (7) tiie training gap, 
(8) cost effectiveness, (9) MDTA as a bridge 



between industry and education, (10) new occupa-- 
tions, (11) public relations and community liaison, 

(12) MDTA contribution to local training effort, 

(13) the Keene, New Hampshire project, (14) edu- 
cational innovation, (15) some problems. It was 
recoinniended tihat jn integrated declslon-iTiaklng 
system in the occupational training area be de- 
veloped to Insure that the total occupational train- 
ing effort of the Federal Government achieves 
the worthy goal of training all individuals to their 
full capacity, 

A BENEFIT-COST ANALYSIS OF THE NEIGH- 
BORHOOD YOUTH CORPS: THE OUT-OF- 

SCHOOL PROGRAM IN INDIANA. Borus, Michael 
E.; and Others, In Hum.an Resources; v5 n2 
p. 139-159. Spr. 1970, 

Neighborhood Youth Corps (NYC) out-of- school 
progranis in Indiana were studied to determine 
benefits to society from training and employment 
efforts— increased aggregate output and increased 
employability of peo-tlclpants. Benefits in enroll- 
ing male participants were considerably higher 
than those derived from enrolling females; ratio 
was at least two to one for males and less than 
one for females. Assuming the saine costs for 
participants, benefits from enrolling trainees with 
less dian nine years of educati.on were substan- 
tially greater than training more educated en- 
rollees. For participants, benefits varied with 
the duration of enrollment; for males and some 
females, die longer participation the greater was 
the increase In post-program earnings. These 
findings differ slgtriificantly from previously pub- 
lished studies. Also, contrary to earlier find- 
ings, males gained more from participation than 
did females. This conclusion may, however, be 
due to differing drsmographic characteristics of 
die enrollees. Furthermore, although a correla- 
tion is assumed be tween length of participation and 
derived benefits, perhaps those qualities which 
were associated witii an individual's longer par- 
ticipation in the program were also related to his 
increased earnings. 

HOW FARE MDTA EX-TRAINEES? AN EIGHT- 
EEN MONTHS FOLLOW-UP STUDY OF FIVE 
HUNDRED SUCH PERSONS, 219 p. Dec. 1967. 
London, H, H, Missouri Unlv„ Columbia Office 
of Manpower Polity, Evaluation, and Research 
(DOL), Washington, D.C, OEC-81-24-25. Avail- 
able from NTIS (Order No. FB 177626, MF 650, 
HC $3.00). 

This study examined the socioeconomic back- 
ground characteristics of Manpower Development 
and Training Act ex- trainees, the socioeconomic 
changes diat occurred after training, ihelong-and 
short-range effectiveness of die services received 
under die MDTA program, and how diese may be 
O ' proved. For approximately 500 persons who 
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completed an MDTA program in St. Louis, Kansas 
City, and Joplin, Missouri, from October 1, 1964 
dirougli September 30, 1965, followup interviews 
were held at 6-, 12-, and 18-month Intervals, 
and employers rated dieir performance following 
each of the three interviews, Additional data 
were collected from the records of relevant state 
and local public agencies. Some findings were: 
(1) 21.4 per cent of the ex- trainees were receiv- 
ing some form of welfare for an average of 14,8 
months some time during the 18 months preced- 
ing training, (2) Pretraining work history was 
both erratic and Irregular, (3) Personal service, 
semiskilled, skilled, and unskilled, in that order, 
together with clerical, accounted for the majority 
of post-training jobs, and (4) Neither the type of 
job nor the wages received seemed to have any 
appreciable bearing on attitude toward the job, 
and most liked dieir work very well or fairly 
well. Findings led to 10 major conclusions and 
six recommendations for improvement of MDTA 
programs. Copies of the data- gathering instru- 
ments used and 62 detailed statistical tables are 
included. 

A BENEFIT COST ANALYSIS OF THE SOUTH 
CAROLINA MDTA PROCRAM. PRELIMINARY 
REPORT. 64 p. Soutiti Carolina Univ„ Columbia, 
Bureau of Business and Economic Research, ED 
029 980, EDRS PRICE MF 650, HC $3.29. Also 
available from the Bureau of Business and Eco- 
nomic Research, University of Soudi Carolina, 
Columbia, South Carolina 29208, (without charge). 

Manpower Development and Training Act 
(MDTA)' programs completed in 1965 were evalu- 
ated by benefit- cost: ratio and internal rate of re- 
turn. Initial annual earnings differential figures 
at $525,650 and $719,629 were projected into the 
future at various rates of promotion and various 
rates of discount on die benefit stream. Result- 
ing lifetime benefits (for 30 working years) range 
from $2,556,110 at a zero promotion rate and 
20 per cent discount rate on the first figure to 
$15,162,977 at a 3.5 promotion rate and a 5 per 
cent discount rate for die $719,629 figure. These 
projected benefits may be compared widi the total 
program cost of $3,097,616, On the other hand, 
die Internal rates of return on the training outlay 
of $3,097,616 range from 16.29 per cent at zero 
rate of promotion for die $525,650 figure to 26.20 
per cent at the 3.5 rate of promotion for die 
$719,629 figure. An alternative for the cost out- 
lay would have to exceed die rate of return If the 
training were to be considered economically un- 
profitable, It was concluded diat, under the cir- 
cumstances and assumptions of die study, train- 
ing conducted under MDTA is worthwhile with 
benefits greatly exceeding costs and Internal rates 
of return high. Assessments of individual pro- 
grams are also provided. 

28 
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BENE FIT-COSI ANALYSIS OF TAT PHASE I 
WORKER TRAINING. TkAINING AND TECH- 
NOLOGY PROJECT. SPECIAL REPORT. Kirby, 
Frederick C.*, Castagna, Paul A. 34 p, July 1969. 
Atomic Energy Commission, Oak Ridge, Tenn. 
ED 034 039, EDRS PRICE MF 650, HC $3.29. 

The purpose of this study is to estimate 
costs and benefits and to compute alternative 
benefit-cost ratios for both the individuals and 
the Federal Government as a result of Investing 
time and resources in the Training and Technology 
(TAT) Project, TAT is a continuing experimental 
program in training skilled workers for private 
Industry. The five occupational areas Included 
in the study are mechanical drafting, welding, 
machining: industrial electronics, and physical 
testing- quality control. Data were obtained by 
analyzing two samples of 70 persons each; one 
sample selected from 407 TAT trainees, the other 
sample from 1,500 qualified applicants not ad- 
mitted because of limited space. The method- 
ology, the cost of training to the Federal Govern- 
ment, the expected value of income differences 
between the trained and control groups by year, 
the expected value of tax differences accruing to 
the Federal Government by year, and the compu- 
tation of the number of deductions per pay check 
are appended. 

CONTRIBUTIONS AND COSTS OF MANPOWER 
DEVELOPMENT AND TRAINING. POLICY 
PAPERS IN HUMAN RESOURCES AND INDUS- 
TRIAL RELATIONS, NO, 5, Mangum, Garth L, 
Michigan Unlv. 95 p. Dec, 1967. ED 021 949, 
EDRS PRICE MF 650, HC $3.29. Document also 
available from Publications Office, Institute of 
Labor and Industrial Relations, Museums Annex, 
Ann Arbor, Michigan 48104 ($2,00), 

As a part of a larger project under grant 
from the Ford Foundation to evaluate federal 
manpower policies and programs, ±ls evaluation 
of the training efforts under Title II of die Man- 
power Development and Training Act (MDTA) 
made use of data provided by the Departments of 
Labor and Health, Education, and Welfare, By the, 
end of fiscal 1967, 1.2 billion dollars had been 
obligated for training the unemployed and under- 
employed under Title II of the Act, As the com- 
bined result of original congressional intent and 
subsequent experience, MDTA induced programs 
have come to have six potential and identifiable 
objectives; (1) facilitating employment of die 
unemployed, (2) reducing poverty, (3) lessening 
inflationary pressures, (4) meeting labor short- 
ages, (5) upgrading the labor force, and (6) re- 
vamping traditional institutions. On the basis of 
a detailed analysis of bodi quantifiable and non- 
quantiflable accomplishments, the extent to which 
each objective has been achieved is examined 
estimates of the overall costs of the program 



are made before turning to a review of cost- 
benefit studies for comparison with previous con- 
elusions. Tlie results of die appraisal are cleaiTy 
favorable. In general, every current component 
may not pay, but die overall contributions of the 
program have exceeded its costs by a margin 
which not only merits support but justifies ex- 
pansion. 

A BENEFIT COST ANALYSIS OF MANPOWER 
RETRAINING (In Proceedings of the 17th Annual 
Mee tiag, Industrial Relations Research Associa- 
tion, Chicago, Dec. 28-29, 1964, pp. 172-85). 
Somers, Gerald G., and Stromsdorder, Ernst W, 
16 p. 1965. 

In a smdy (1962-64) designed to evaluate bene- 
fits and costs of manpower training, 1379 West 
Virginia workers were surveyed— 501 Area Re- 
development Act or Area Vocational Training 
Program trainees, 233 program dropouts, 65 who 
were accepted but did not report, 127 “rejects*’, 
and 453 randomly chosen nontrainees (the control 
group). By the summer of 1962, 60 per cent of 
trainees were employed versus 56 per cent of the 
dropouts and about a third of the others, A year 
later, employment rates were up for all groups, 
with 71 per cent for trainees and slightly less for 
other groups. About 12 per cent of successful 
trainees were out of the labor force (Summer 
1962), with similar percentages among the odiers. 
Trainees consistently had higher rates of em- 
ployment than others of comparable age, educa- 
tional level, or employment background. Between 
70 and 75 per cent of trainees have found jobs, 
almost all in training related occupations, and 
earnings (especially among men) have substan- 
tially increased. Payback periods (time required 
for differential earning gains to offset total rt- 
tralnlng costs) have tended to be short, especially 
for men, and shorter for individuals than for so- 
ciety, Overall results indicate that -the benefit 
and potential benefits of manpower retraining will 
greatly outweigh costs regardless of local cir- 
cumstances. Document Includes foomotes and 
3 tables. 

DETERME4ANTS OF RATES OF RETURN TO 
INVESTMENT IN ON-THE-JOB TRAINING, Ras- 
mussen, Dale Bruce, Soutiiem Mediodlst Unlv. 
159 p. Mar. 1969. Available from NTIS (PB- 
184-024, MF 650, HC $3.00). Ph.D. Thesis. 

The study evaluates public investtnent in on- 
tiie-job training (OJT) and tries to identify sys- 
tematic determinants of the profitability of this 
investtnent. Profiles of age and earnings by 
occupation are estimated for the Soutii and odier 
regions, and for whites and non- whites, from data 
in die one-ln- 1,000 census sample and are dien 
adjusted to the regional level using wage survey 
indexes. Concepts of profitability reflect three 



kinds of assumptions: (1) transfers (government 
expenditures for allowances to trainees) are not a 
cost; (2) transfers are a cost; (3) Increases in 
tax revenue are the sole benefit derived from 
OJT. The methodology of the study is an aggre- 
gate approach to evaluating investment by utiliz- 
ing detailed Information describing individuals. 

THE ECONOMIC EFFECTIVENESS OF ON-THE- 
JOB TRAINING: THE EXPERIENCE OF THE 

BUREAU OF INDIAN AFFAIRS IN OKLAHOMA. 
Scott, Loren C. In Industrial andLabo r Relations 
Review; v23 n2 p. 220-236. Jan. 1970. 

In 1958 the Bureau of Indian Affairs (BIA) 
instituted an on-the-job training program (OJT) 
in which Indian participants could Increase their 
skills through work experience. As an incentive 
to firms to undertake this training, apaitial wage 
subsidy was provided. Indians desiring OJT were 
screened by the Bureau and then referred to the 
firm for employment and training. Information 
obtained from participating employers suggests 
that the lengdi of training periods is generally too 
long. A theoretical framework for benefit, cost 
analysis of the program was developed along the 
following dim.ension: (1) private benefits; (2) so- 
cial benefits; (3) private costs; and (4) social 
costs. The private, explicit benefits are reflected 
in changes in earning and employment experiences 
of the trainees. These benefits are Impressive 
since trainees earned an average of $125 more 
per month after participation. Due to the rela- 
tively small number of participants entering any 
one occupation, it Is unlikely that there was any 
significant change in the labor supply in that field. 
Though there were no private costs to the trainee, 
taxpayers did bear the expense of providing the 
training. Data are also presented in this article 
regarding completion of training, characteristics 
of the trainees, and productivity differentials. 

RETURNS FROM INVESTMENT IN SCHOOLING 
AND ON-THE-JOB TRAINING. Johnson, Thomas. 
133 p. 1969. Nordi Carolina State University at 
Raleigh. University Microfilms, (Order No, 70- 
9191, MF $3.00, Xerography $6.40). Ph.D. Thesis. 

This study was made to develop a model 
which will predict lifetime earnings and to use 
this model to Investigate the relation between 
schooling and on-the-job training (OJT) as afunc- 
tlon of race and regions, trying to distinguish 
earnings differences due to differences in the re- 
turn on investments made and differences in 
opportunities to make invesmients in OJ i. The 
model was developed by representing an Indlvld- 
ual*s Investment in human . capital as a continuous 
process, ralher than a series of discrete invest- 
ments. The data employed were the earnings 
profiles by race and region which Hanoch (1965 

and 1967) estimated from the I960 census one 
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in- 1,000 sample survey. Results Indicated that, 
for most levels of schooling, it cannot be shown 
that non-whites received a lower rate of return 
than whites; and that for most of the region- 
schooling groups, nonwhites also invest at least 
as large a fraction of their earning capacity in 
OJT as do whites. It was concluded that the model 
developed can well predict the average earnings 
streams of broad aggregates of persons. How- 
ever, problems were met in data limitations on 
estimation of base earnings capacity and tliere 
was a problem of identifying discrimination. It 
is of interest that marginal market discrimina- 
tion was indicated at only a few schooling or skill 
levels. 

Training— General 

A GENERALIZED NETWORK MODEL FOR 
TRAINING AND RECRUITING DECISIONS IN 
MANPOWER PLANNING. Charnes, A.; and 
Others. Carnegle-Mellon Univ, 16 p. May 1970. 
Available from NTIS (AD-7 11- 8 36, MF 650, HC 
$ 3 . 00 ). 

Models for manpower planning previously 
devised for the U.S. Navy’s Office of Civilian 
Manpower Management have all utilized goal pro- 
gramming constructs with embedded Markoff 
processes. These models — referred to as ‘OCMM 
Models’ — are here extended to include training 
elements along with related constraints. 

COSTS AND BENEFITS OF ADULT RETRAINING 
IN THE UNITED KINGDOM. Zlderman, A. In 
' feconomlca; v36 nl44 p. 363-376, Nov. 1969. 

Indirect effects of retraining on die resources 
of the economy include an increase in the number 
of skilled workers in an area which suffers from 
a shortage of qualified; but if retrained workers 
merely displace odier, less qualified, workers 
who join the unemployed, then there is no benefit 
to the economy, A relatively accurate and com- 
monly used method for measuring the economic 
benefits of retraining is to compare die perform- 
ance over a period of time of a group of retrained 
workers widi a control group who were not 
trained. This method fails to take into account 
the Inherent qualities of the workers chosen for 
retraining. The situation is aggravated if (as in 
Great Britain) retraining is given to employed 
workers as well as to the unemployed. Despite 
the complexity of the retraining cycle, it is possi- 
ble to calculate the over- all return to society of 
investing resources in government programs. On 
the basis of figures for the year 1966, a table has 
been drawn up which shows costs, benefits, aver- 
age pay-back period, and rate of return. The 
drawback Is diaL tiiese figures do not take into 
account the Indirect effects (vacancies created 
by workers being trained, retrained workers 
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displacing others, etc.). The method is useful but 
the availability of data on the indirect effects 
might well produce different results. The results 
can, therefore, only be considered indicative of 
the positive benefits of retraining, 

TRAINING COSTS WITH REFERENCE TO THE 
INDUSTRIAL TRAINING ACT. Garbutt, Douglas. 
210 p. 1969. Available from Gee and Company 
(Publishers) Limited, 151 Strand, London W.C. 2, 
England (52s 6d). 

Provisions and implications of the British 
Industrial Training Act of 1964 (including the sys- 
tem of training grants and levies) are set forth. 
Procedures for accounting and budgeting for 
training costs, routines for collecting training 
information, documents (budgets, cost sheets, 
control statements) for collecting and controlling 
costs, means of estimating expenditures for train- 
ing activities, and steps in planning the training 
function for optimum costs and benefits are also 
discussed. The point is stressed that, while data 
on how much training is done by a firm must be 
gathered if grants are to be obtained, relatively 
minor adjustments from previous mediods of data 
gathering will suffice. Personnel payments and 
fringe benefits, overhead, production, insurance, 
administration, transportation and travel, depre- 
ciation, materials, and other cost categories are 
suggested for calculating training costs. The 
document also includes a diagram of the indus- 
trial training system, based on cooperation by 
industry with the Departments of Tfechnology and 
Labour; descriptions of a graduate training and a 
student trainee scheme; lists of statutory instru- 
ments affecting the Civil Air Transport Board and 
the engineering industry; and 78 other figures, 

OPERATOR TRAINING IN INDUSTRY, Seymour, 
W. Douglas, 55 p. 1965. Institute of Personnel 
Management, London (England). Avallable^from 
Institute of Personnel Management, 80 Fetter 
Lane, London E.C. 4, England (7s 6d). 

Guidelines and objectives are set fortii for 
more effective training of machine operators and 
other semiskilled and unskilled Industrlai work- 
ers, Basic kinetic and perceptual elements of 
manual skill development are briefly reviewed. 
Operative training Itself is analyzed under die 
following headings: handwork widiout tools ; hand- 
work with tools; single purpose machine work; 
multiple purpose machine work; group machine 
work (as in cotton weaving); and nonrepetltive 
work involved in such activities as Inspection 
and steelmaking. Nine figures and 24 references 
are included. 

CONCEPTUAL ISSUES IN EVALUATING TRAIN- 
ING PROGRAMS, Weisbrod, Burton A. In 
Mondily Labor Review , v89 nlO, October 1966. 
7 p, Oct. 1966, 



Based on the assumption that training pro- 
grams ordinarily involve both benefits (adva 
tages) and costs (disadvantages), this article dis- 
cusses the form of such effects in the manpower 
training area, and presents some suggestions re- 
garding measurement methods. The need to 
evaluate training programs in terms of their 
effects on efficiency of resource allocation and 
on the distribution of Income is underscored. 

C-130 PHASE I PILOT TRAINING PROGRAM 
(CCTS). Shaw, Jack B.; and Others. 172 p. Apr. 
1969. Tactical Airlift Center, Pope AFB, North 
Carolina. Available from NTIS (AD-700-987, 
MF 650, HC $3.00). 

This is a detailed study of the C-130 Phase I 
(CCTS) pilot training program conducted by the 
442nd Combat Crew Training Wing at Stewart 
Air Force Base, Tennessee. The purpose of the 
study was to determine the most effective training 
program that will continue to produce highly quali- 
fied pilots at the least cost. The program was 
evaluated and compared with otlier representative 
military and civilian training programs. Analysis 
of the program was concerned widi increases in 
graduate quality and simultaneous decreases in 
student pipeline time and training costs. Changes 
proposed include revision of existing program 
using the system approach and programmed learn- 
ing in the course curriculum. Portions of this 
document not fully legible. 

MEASURING THE EFFECTIVENESS OF SALES 
TRAINING. Gillam, A, In Marketing World, 
p. 14-17. May, 1970, 

There is a need to look at sales training 
afresh, to re-examine the objectives. Simple 
yardsticks for measuring salesmen performance 
are: number of calls per day; number of orders 

per day; percentage of orders to calls. These 
apply only to comparatively simple sales situa- 
tions, A more thorough and complex investiga- 
tion would: define responsibilities and job speci- 
fications of the sales organization; examine diem 
in relation to the company’s sales objectives; 
examine records and controls; see how sales- 
men’s reports and orders are handled in the sales 
office; establish true objectives; and record cur- 
rent performance levels of salesmen. The extent 
to which results can be measured will depend 
very much on title nature of the particular sales 
operations; but some attempt should be made to 
put a finite measure on training. Industrial train- 
ing boards have given a big boost to salesman 
training, and new techniques built around the com- 
pany’s specific sales policy are being developed. 
However, before an Industrial Training Board will 
part wltih its money, it will have t( be persuaded 
that sales training really works. 



CONTROL OF THE TRAINING FUNCTION IN 
THE ENGINEERING INDUSTRY. Baker, H. A. 
In Training Officer; v6 nl, p, 14-17. Jan, 1970, 

Levy and grant provisions under the Engi- 
neering Industry Training Board in Great Britain 
are discussed as a basis for procedures in analys- 
ing and reporting on training costs and benefits, 

COST CONTROL OF TRAINING — DOES IT MAKE 
SENSE? Longley, James. In Industrial Training 
International; v4 nil, p. 471-474. Nov, 1969. 

THE TRAINING FUNCTION: OVERHEAD OR 

PROFIT? Livingston, Henry S. In Training and 
Development Journal; v24 nS, p, 18-24, Aug. 1970. 

PLANNING INDUSTRIAL TRAINING. Evans, 
Eric; and Others, 46 p, Nov, 1968. National 
Institute of Adult Education (England) London. 
Available from National Institute of Adult Educa~ 
tion, 35 Queen Anne St., London W. 1, England 
(10s 6d), 

Under the pressure of economic conditions 
and the stimulus of the Industrial Training Act, 
industrial and commercial firms in Great Britain 
are acting more and more In die belief that train- 
ing is Inseparable from the mainstream of their 
activities. This pamphlet is concerned with Issues 
of practical importance in this context: the rela- 
tive importance of training directed to different 
levels of skill; prediction of likely performance as 
a basis for selection of trainees; die role of 
training officers, in higher decision making; and 
techniques for evaluating the return on training 
expenditures. Included are chapter references, 
an interview form, a training system outline, 
cost effectiveness curves, and statistical data on 
labor force skills, occupational and employment 
trends, recruitment and training expenditures, 
and course results from two experimental pre- 
apprentice programs. 

COST-BENEFIT ASPECTS OF MANPOWER RE- 
TRAINING. MANPOWER PAPERS NO. 2, 42 p, 
1970. Great Britain. Dept, of Employment and 
Productivity, London (England). Available from 
Her Majesty*s Stationery Office, London, England, 
(4s 6d). 

This study brings togedier die extensive 
literature on the subject and discusses die need 
for, and advantages of, retraining; die provisions 
for retraining made in die United States and 
Sweden; and possible approaches to cost-benefit 
analysis of retraining. For die Individual, man- 
power retraining is well worthwhile, since It leads 
to considerable gains in employment and income. 
For die government and the economy, die studies 
revealed that investment in manpower retraining 
is even more wordiwhlle, aldiough the exact mag- 
nitude of the benefit- cost ratios is difficult to 
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estimate because of the existence of externalities. 
Ml jpower retraining can help alleviate resource 
bottlenecks if directed toward areas of skill short- 
age. This will not only help in the fight against 
inflation; it will also facilitate a more general 
expansion of die economy. In a tight market, 
those who remain unemployed will increasingly 
represent the ^‘hard-core” unemployed. The 
object of the program should be to Increase their 
employability so that they can be reabsorbed 
into die system. In a slack labor market, job 
vacancies can stiU be filled, especially if retrain- 
ing courses are geared to the needs of individual 
employers. Ho wever, it is in a depressed market 
that large-scale preparations can be made for 
future expansion. Furdier research is required 
to compare the benefit- cost ratios of instimtional 
training with those of on-the-job training. 

TECHNICAL PROGRESS AND TECHNICAL 
SCHOOLS. Pejovich, Svetozar. In Review of 
Social Economy ; v26 nl, p. 40-49. Mar. 1968, 
Cost effectiveness analysis of the p ovlsion 
of training or retraining for those whose job 
stability is threatened by technical progress may 
suggest die superiority of specifically oriented 
on- die- job training to more generally oriented 
technical school training, contrary to first im- 
pression. The possibility is suggested that train- 
ing in technical schools may provide the trainee 
with a higher income, but that on-the-job training 
may provide the trainee with greater assurance 
against cyclical unemployment. If this is true, 
dien government investment in training programs 
should not be based on the assumption that the 
trainees would prefer higher Income to the less- 
ened likelihood of unemplo^mient. Investment in 
research to arrive at a better dieoretlcal under- 
standing of which choice the affected individuals 
would make is better merited than Investment in 
trial- and- error retraining programs. 

MANPOWER REPORT OF THE PRESIDENT, IN- 
CLUDING A REPORT ON MANPOWER REQUIRE- 
MENTS, RESOURCES, UTILIZATION, AND 
TRAINING. STATISTICS ON MANPOWER. A 
SUPPLEMENT TO THE MANPOWER REPORT 
OF THE PRESIDENT (March 1961), 372p. Jan. 
1969. U.S. Dept, of Labor,^ Washington, D.C, 
ED 032 392, EDRS PRICE MF 650, HC available 
from U.S. Government Printing Office, (L. 1.42/ 2- 
2:969, $1.00; L,1.42/2;969, $2.00). 

The seventh annual manpower report is pre- 
sented in three parts: The Employment Record, 
Manpower Policy and Programs, and Manpower 
Research and Experimentation. Discussion of 
the employment record addresses emplo 3 anent de- 
velopments and their economic background, pat- 
terns of employment growth, unemployment and 
underemployment, developments in the labor 
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force, and productivity. Sub-divisions of the sec- 
tion on manpower policies and programs include 
manpower development chrough the schools, en- 
listing private industry cooperation, and meeting 
individual needs. The focus of manpower re- 
search was on manpower requirements and re- 
sources, joblessness, occupational training, job 
market processes, and equal employment oppor- 
tunity. Experimental and demonstration programs 
are discussed relative to program accom. .ish- 
ments, impact on legislation, and inter-agency 
cooperation, A cost-effectiveness analysis of 
manpower programs, an evaluation of manpower 
programs, and statistical tables on the labor 
force, employment, unemployment and training 
are appended, 

RETRAINING THE UNEMPLOYED. Somers, 
Gerald G„ ed, 350 p. 1968. Available from 
University of Wisconsin Press, Box 1379, Madi- 
son, Wisconsin 53701 ($10.00). 

Case histories evaluate the effectiveness of 
recent and current retraining programs for un- 
employed workers under Federal, state, muni- 
cipal, and union- management auspices, Judging 
them by their success in placing workers in use- 
ful employment in distressed economic areas. 
Data on various aspects of retraining programs 
under a wide variety of circUi. stances have been 
assembled, and retralnees are compared with 
such control groups as re jected applicants, train- 
ees who dropped out before completing their 
courses, and unemployed workers who did not 
apply for retraining. Through cost benefit analy- 
ses and other evaluation of specific programs in 
West Virginia, Connecticut, Massachusetts, Mich- 
igan, Tennessee, and soudiem Illinois, and the 
cities of Chicago, Fort Worth, and Omaha, where 
such factors as the personal characteristics of 
trainees, tihe labor market. Institutional place- 
msnt, and procedural arrangements varied widely, 
significant conclusions have been drawn as to the 
economic and other benefits of retraining pro- 
grams in general. The document includes chapter 
references, appendixes, and 114 tables and figures. 

LEVY AND GRANT SCHEMES: THEIR CONTRI- 
BUTION TO THE ACHIEVEMENT OF THE IN- 
DUSTRIAL TRAINING ACT. Dawson, Peter. In 
Industrial Training International ; v3 nlO, p. 471- 
3. Oct, 1968. 

There is growing questioning of the contribu- 
tion of levy and grant schemes to the achieve- 
ment of the aims of die Industrial Training Act 
of Great Britain to enable decisions on the scale 
of training as related to economic needs and tech- 
nological developments, to improve die overall 
quality of training, and to spread die cost more 
fairly. A simpler and cheaper approach would 
be for boards to make grants for all employees 
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who left firms and who had attended approved 
training during the 12 months before they left 
and for levies to fall on firms who were recruit- 
ing trained manpower and deriving the benefit 
from other firms’ training efforts. 

EXPERIENCE ON-THE-JOB AND FORMAL 
TRAINING AS ALTERNATIVE MEANS OF SKILL 
ACQUISITION: AN EMPIRICAL STUDY. Maton, 
J. In International Labour Revlev; : vlOO n3, p. 
239-255. Sep. 1969. 

On the results of a survey carried out among 
ten undertakings in Belgium and six in Argentina 
with a sample of 200 to 300 skilled workerSj mid- 
dle-level technicians and assistant engineers, a 
comparative study was made of the time requlre- 
ments for various combinations of training = 
formal training and on- die- job experience. The 
primary school graduate has the choice between 
zero and six years of technical secondary educa- 
tion, Each additional year of such training will 
exempt him from a period of experience on the 
job. In this way, the possible combinations neces- 
sary to become a tool and die-maker can be de- 
termined; these combinations can be represented 
by Curves of possible choices. Using die results 
of die survey, scatter diagrams of the combina- 
tions were plotted for each category of worker. 
Using regression analysis, representative em- 
pirical curves and possible training combinations 
were determined. The analysis of these curves 
leads to a set of training equivalencies (in years) 
set out in tabular form. It was found that, if the 
costs and loss of practical experience while under- 
going formal training are taken into account, it 
can be stated that in conditions favorable to skill 
diffusion, selected workers may become highly 
skilled trough practical ej^erlence equally well 
as through formal training. The Introduction of 
die senior cycle of secondary education shifts 
the training to the school and its financing to the 
government from industry. 

TRAINING FOR RESULTS: A SYSTEMS AP- 

PROACH TO THE DEVELOPMENT OF HUMAN 
RESOURCES IN INDUSTRY, Warren, Malcolm 
W, 249 p. 1969. Available from Addison- Wesley 
Publishing Company, Inc,, Reading, Massachu- 
setts 01867. 

Using die systems approach, dils book ex- 
plores training in Industry as an. organization 
tool for developinghummi resources, and stresses 
training results and return on iiivestment radier 
dian training programs as such. It is Intended as 
a guide for general managers, personnel and in- 
dustrial relations directors, and training direc- 
tors on how training functions, how and when it 
is cost effective, how to evaluate it, and how It 
should be structured wldiin the mass production, 
sales, supervisory, administrative, and technical 
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ti'aining fields. Included ai'e sections on cost 
estimating and evaluating training in terms of 
objectives and results. Training systems ele- 
ments (research, analysis, development, opera- 
tions, evaluation) are also explained, together 
with such other aspects as instructional design, 
feedback, financial support for employee self- 
improvemeiit, and tiie organization and staffing 
of the training function. An index, bibliography, 
and 19 tables and figures are also included. 

CAN WE EVALUATE TRAINING EXPENDI- 
TURES? Barrett, James E. In Training in Busi- 
ness and Industry ; v7 nl, p. 40-43, Jan. 1970. 

Although accurate measurement of training 
effectiveness is expensive and time consuming, 
it is possible. Many formal measurements of 
training results and a few formal measuiements 
of training value have been created. 

PROFIT FROM TRAINING. Hennessy, David E. 
In Training in Business and Industry ; v6n2, Feb. 
1969. 

Training efforts In business corporations 
often never emerge from the concept stag ; be- 
cause of the failure of training directors to speak 
the language of upper management: dollars and 
cents. A method of analyzing and presenting such 
training in terms of increasing corporate analyz- 
ing and presenting such training in terms of in- 
creasing corporate profits by reducing peroonnel 
deficiencies is elucidated and involves the follow- 
ing steps: (1) determine where personnel defi- 

ciencies exist which are contributing to cashout- 
flow and how much money could be saved If fhe 
deficiencies were 100% eliminated; (2) determh 3 
the training program effectiveness in terms of 
percentage of such and dollar equivalent, and ^e 
allowable training prograni cost; (3) determine 
the actual training program development cost 



and the expected effectiveness level; (4) prepare 
a cash-flow sheet and diagram before making the 
budget program; (5) after project approval, de- 
velop and evaluate pilot programs in the field 
before proceeding to implement a full scale pro- 
gram. 

AGCOUNTING FOR PERSONNEL TRAINING AND 
development COSTS. Douthout, John, In 
Training and Development Journal ; v24n6, p.2-6. 
June 1970, 

Conventional accounting practices tend to 
penalize managers who Invest in human resource 
development. Five tables and figures. Four 
references. 

TEACHING FUTURE SALES MANAGERS HOW 
TO MANAGE: A METHOD FOR TEACHING 

SALES ORGANIZING, PLANNING, AND CON- 
TROLLING. Dies, Robert P, In Training and 
DeveloDme nt Journal ; v23 nl, p. 24-6. Jan. 1969. 

' Educational techniques were developed to 
help line executives train first-line supervisors 
in modern sales management theory and practice, 
A first attempt at critique and feedback resulted 
In the need for planning a morL basic program: 
theory, concept clarification, practical applica- 
tion, and evaluation. Training; specialists con- 
ducted group sessions on theory and presented a 
team test of sales management,, Regional manag- 
ers led the following session on test correction 
and practical application of concepts. During the 
interval between training sessions the men applied 
learnings to their own work and prepared presen- 
tations of their results. The next training session 
revealed excellent analysisof zones sales results, 
confident and enthusiastic trainees, and develop- 
ment of individual sales planning, organizing and 
controlling programs. 



Vlt NOTE ON 

Many documents listed in &ls bibliography 
are available in microfiche (MF) and hard copy 
(HC) reproductions from die ERIC Document 
Reproduction Service, (Microfiche Is a reproduc- 
tion on a 4x6 Inch sheet of film with up to 70 
images, each representing a page of die original 
document. The titles are eye-readabla, but a 
standard microfiche reader is requires to en- 
large the Images for reading purposes.) 

Only diose documentsldentifled wlthan “ED 
number may be ordered from EDRS. All o^er 
documents must be obtained from dieir original 
sources, or from other agencies making them 
available as noted in the citations. 
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The “ED" number (tide need not be given); 
The type of reproduction desired — micro- 



fiche or hard copy; 

The numb«jr of copies being ordered. 



Payment must accompany orders totalling less 
than $10. 
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